yard 0.9.28 → 0.9.29

data/docs/Handlers.md

@@ -1,152 +0,0 @@
-# @title Handlers Architecture
-
-# Handlers Architecture
-
-Handlers allow the processing of parsed source code. Handling is done after
-parsing to abstract away the implementation details of lexical and semantic
-analysis on source and to only deal with the logic regarding recognizing
-source statements as {file:docs/CodeObjects.md code objects}.
-
-![Handlers Architecture Class Diagram](images/handlers-class-diagram.png)
-
-## The Pipeline
-
-After the {file:docs/Parser.md parser component} finishes analyzing the
-source, it is handed off for post-processing to the {YARD::Handlers::Processor}
-class, which is responsible for traversing the set of statements given by
-the parser and delegating them to matching handlers. Handlers match when the
-{YARD::Handlers::Base.handles?} method returns true for a given statement.
-The handler can then perform any action after being invoked by the `process`
-method.
-
-## The Processor Class
-
-The main purpose of the processor, as mentioned above, is to traverse through
-the list of statements given to it by the parser. The processor also keeps
-state about what is being processed. For instance, the processor is what keeps
-track of the current namespace (the module or class an object is being defined
-in), scope (class or instance), file and owner. The owner refers to the object
-that is most directly responsible for the source statement being processed. This
-is most often the same as the namespace, except when parsing the body of a method,
-where the namespace would be the class/module the method is defined in and the
-owner would be the method object itself.
-
-## Implementing a Handler
-
-This section covers the basics of implementing a *new-style* Ruby handler. For
-details on implementing a legacy handler, see the "API Differences" section below.
-
-a Ruby handler can be implemented simply by subclassing the {YARD::Handlers::Ruby::Base}
-class and declaring what node types or source to process with the {YARD::Handlers::Base.handles handles}
-class method. A very simple handler that handles a module definition would be:
-
-    class MyModuleHandler < YARD::Handlers::Ruby::Base
-      handles :module
-
-      def process
-        puts "Handling a module named #{statement[0].source}"
-      end
-    end
-
-For details on what nodes are, and what node types are, see the
-{file:docs/Parser.md parser architecture document}.
-
-In this case the node type being handled is the `:module` type. More than one
-node type or `handles` declarations may describe a single handler, for instance,
-a handler that handles class definitions should handle the `:class` and `:sclass`
-node types respectively (the latter refers to classes defined as `class << Something`).
-The {YARD::Handlers::Base#statement statement} attribute refers to the current
-node (or statement) that is being handled by the handler.
-
-### Handling a Method Call
-
-In some cases, a developer might need to handle a method call. The parser can
-express a method call in many AST forms, so to simplify this process, a method
-call can be handled by declaring the following in a `handles` statement:
-
-    class MyHandler < YARD::Handlers::Ruby::Base
-      handles method_call(:describe)
-
-      def process
-        # Process the method call
-      end
-    end
-
-In this case we handle any of the method calls to method name `describe` with
-the following syntaxes:
-
-    describe(something)
-    describe arg1, arg2, arg3
-    describe(something) { perform_a_block }
-    describe "Something" do
-      a_block
-    end
-
-### Creating a new Code Object
-
-Usually (but not always) handling is performed to create new code objects to add
-to the registry (for information about code objects, see {file:docs/CodeObjects.md this document}).
-Code objects should simply be created and added to the existing `namespace`. This
-will be enough to add them to the registry. There is also a convenience
-{YARD::Handlers::Base#register register} method which quickly sets standard attributed
-on the newly created object, such as the file, line, source and docstring of the
-object. This method will be seen in the next example.
-
-### Handling an Inner Block
-
-By default, the parser gives the processor class a list of all the top level
-statements and the processor parses only those top level statements. If an inner
-block of a module, class, method declaration or even a block passed to a method call
-needs to be handled, the {YARD::Handlers::Base#parse_block parse_block} method must be called on the list of statements
-to parse. This will send the list to the processor to continue processing on that
-statement list. The source tree can be selectively parsed in this manner by parsing
-only the inner blocks that are relevant to documentation.
-
-For example, the module handler parses the inner body of a module by performing
-the following commands:
-
-    class YARD::Handlers::Ruby::ModuleHandler < YARD::Handlers::Ruby::Base
-      handles :module
-
-      def process
-        modname = statement[0].source
-        mod = register ModuleObject.new(namespace, modname)
-        parse_block(statement[1], :namespace => mod)
-      end
-    end
-
-In this case `statement[1]` refers to a list of extra statements, the block we
-wish to parse. Note here that when parsing objects like modules and classes,
-we set the namespace for the duration of the block parsing by setting options
-on the `parse_block` method.
-
-### API Differences for Legacy Handler
-
-Because the legacy handler uses the legacy parser and therefore a different kind
-of AST, there are subtle differences in the handler API. Most importantly, the
-`handles` method usually deals with either lexical tokens or source code as a string
-or RegExp object. The statement object, similarly, is made up of lexical tokens instead
-of semantically parsed nodes (this is described in the {file:docs/Parser.md parser document}).
-
-The module example above can be rewritten as a legacy handler as follows:
-
-    class YARD::Handlers::Ruby::Legacy::ModuleHandler < YARD::Handlers::Ruby::Legacy::Base
-      handles TkMODULE
-
-      def process
-        modname = statement.tokens.to_s[/^module\s+(#{NAMESPACEMATCH})/, 1]
-        mod = register ModuleObject.new(namespace, modname)
-        parse_block(:namespace => mod)
-      end
-    end
-
-A few notes on the differences:
-
-  * We inherit from `Legacy::Base` instead of the standard Ruby Base handler class.
-  * We exchange node type `:module` for `TkMODULE`, which represents the
-    first token in the statement.
-  * We perform direct string manipulation to get the module name.
-  * `parse_block` does not take a list of statements. In the old parser API,
-    each statement has a `block` attribute which defines the list of
-    statements within that statement, if any. Therefore, `parse_block` will
-    always parse the `statement.block` if it exists.

data/docs/Overview.md

@@ -1,61 +0,0 @@
-# @title Architecture Overview
-
-# Architecture Overview
-
-YARD is separated in three major components, each of which allows YARD to be
-extended for a separate purpose. The split also emphasizes YARD's design choice
-to explicitly separate data gathering from HTML document generation, something
-that tools like RDoc do not do. These components are:
-
-* [Code Parsing & Processing Component](#parsing)
-* [Data Storage Component](#storage)
-* [Post Processing & Templating System](#templates)
-
-This separation is a major goal of the project, and means that YARD is not *just*
-a tool to generate HTML output. The expectation is that any subset of YARD's
-major components may be used, extended or modified independently. YARD may be
-used just as a data gathering tool (to parse and audit code), just as a data
-source (a webserver containing raw unformatted data about code), or just as a
-conventional HTML documentation generation tool (like RDoc).
-
-The important classes and dependencies of these components are shown in the
-following class diagram:
-
-![Overview Class Diagram](images/overview-class-diagram.png)
-
-<a name="parsing"></a>
-
-## Code Parsing & Processing Component
-
-This component is made up of four sub-components, each of which have separate
-tasks during the data gathering process (*note: the tag architecture is not*
-*shown in the class diagram*). These sub-components are:
-
-  * {file:docs/Parser.md}
-  * {file:docs/Handlers.md}
-  * {file:docs/CodeObjects.md}
-  * {file:docs/Tags.md}
-
-The parser component reads source files and converts it into a set of statements
-which the handlers then process, creating code objects which in turn create tags
-(meta-data) attached to the objects. These objects are all added to the {YARD::Registry},
-the data store component.
-
-<a name="storage"></a>
-
-## Data Storage Component
-
-This component is currently implemented as a simple Ruby marshalled flat namespace
-of object. The implementation is found in the single class {YARD::Registry}, which
-is the centralized repository for all data being parsed, stored and accessed. There
-are future plans to improve this storage mechanism to be backend agnostic and allow
-for more robust storage.
-
-<a name="templates"></a>
-
-## Post Processing & Templating System
-
-This component handles processing of objects from the registry through a templating
-engine that allows output to a variety of formats. Practically speaking, this is
-where templates can be implemented to change the design, output or structure of
-the data. See {file:docs/Templates.md Templates Architecture} for a complete overview.

data/docs/Parser.md

@@ -1,191 +0,0 @@
-# @title Parser Architecture
-
-# Parser Architecture
-
-The parser component of YARD is the first component in the data processing pipeline
-that runs before any handling is done on the source. The parser is meant to translate
-the source into a set of statements that can be understood by the {file:docs/Handlers.md Handlers}
-that run immediately afterwards.
-
-The important classes are described in the class diagram of the entire parser
-system below:
-
-![Parser Class Diagram](images/parser-class-diagram.png)
-
-(Note: the RubyToken classes are omitted from the diagram)
-
-## SourceParser
-
-The main class {YARD::Parser::SourceParser} acts as a factory class, instantiating
-the correct parser class, an implementation of {YARD::Parser::Base}. The selected parser
-is chosen based on either the file extension or by selecting it explicitly (as an argument
-to parsing methods). YARD supports Ruby and C source files, but custom parsers can
-be implemented and registered for various other languages by subclassing `Parser::Base`
-and registering the parser with {YARD::Parser::SourceParser.register_parser_type}.
-
-This factory class should always be used when parsing source files rather than
-the individual parser classes since it initiates the pipeline that runs the
-handlers on the parsed source. The parser used must also match the handlers,
-and this is coordinated by the `SourceParser` class as well.
-
-## Using the SourceParser Class
-
-The `SourceParser` class API is optimized for parsing globs of files. As such,
-the main method to use the class is the `parse` class method, which takes an
-array of file globs or a single file glob.
-
-    YARD::Parser::SourceParser.parse('spec_*.rb')
-    YARD::Parser::SourceParser.parse(['spec_*.rb', '*_helper.rb'])
-
-This is equivalent to the convenience method {YARD.parse}:
-
-    YARD.parse('lib/**/*.rb')
-
-In some cases (ie. for testing), it may be more helpful to parse a string of input
-directly. In such a case, the method {YARD::Parser::SourceParser.parse_string} should be
-used:
-
-    YARD::Parser::SourceParser.parse_string("def method(a, b) end")
-
-You can also provide the parser type explicitly as the second argument:
-
-    # Parses a string of C
-    YARD::Parser::SourceParser.parse_string("int main() { }", :c)
-
-Note that these two methods are aliased as {YARD.parse} and {YARD.parse_string} for
-convenience.
-
-## Implementing and Registering a Custom Parser
-
-To implement a custom parser, subclass {YARD::Parser::Base}. Documentation on which
-abstract methods should be implemented are documented in that class. After the class
-is implemented, it is registered with the {YARD::Parser::SourceParser} factory class
-to be called when a file of the right extension needs to be parsed, or when a user
-selects that parser type explicitly. To register your new parser class, call the
-method {YARD::Parser::SourceParser.register_parser_type}:
-
-    SourceParser.register_parser_type(:my_parser, MyParser, 'my_parser_ext')
-
-The last argument can be a single extension, a list of extensions (Array), a single Regexp, or a
-list of Regexps. Do not include the '.' in the extension.
-
-
-## The Two Ruby Parser Types
-
-When parsing Ruby, the SourceParser can either instantiate the new {YARD::Parser::Ruby::RubyParser}
-class or the {YARD::Parser::Ruby::Legacy::StatementList} class. The first of the
-two, although faster, more robust and more efficient, is only available for
-Ruby 1.9. The legacy parser parser is available in both 1.8.x and 1.9, if
-compatibility is required. The choice of parser will affect which handlers
-ultimately get used, since new handlers can only use the new parser and the
-same requirement applies to the legacy parser & handlers.
-
-## Switching to Legacy Parser
-
-By default, running YARD under Ruby 1.9 will automatically select the new parser
-and new handlers by extension. Although YARD supports both handler styles, plugins
-may choose to only implement one of the two (though this is not recommended). If
-only the legacy handlers are implemented, the `SourceParser` class should force
-the use of the legacy parser by setting the `parser_type` attribute as such:
-
-    YARD::Parser::SourceParser.parser_type = :ruby18
-
-The default value is `:ruby`. Note that this cannot be forced the other way around,
-a parser type of `:ruby` cannot be set under Ruby 1.8.x as the new parser is not
-supported under 1.8.
-
-## RubyParser (the New Parser)
-
-The new Ruby parser uses the Ripper library that is packaged as part of stdlib
-in Ruby 1.9. Because of this, it can generate an AST from a string of Ruby input
-that is similar to the style of other sexp libraries (such as ParseTree). Each
-node generated in the tree is of the base type {YARD::Parser::Ruby::AstNode},
-which has some subclasses for common node types.
-
-### AstNode Basics
-
-The `AstNode` class behaves like a standard Array class in which all of its data
-make up the list of elements in the array. Unlike other sexp style libraries, however,
-the node type is not the first element of the list. Instead, the node type is defined
-by the `#type` method. The following examples show some of the basic uses of `AstNode`:
-
-    # The sexp defines the statement `hello if 1`
-    node = s(:if_mod, s(:int, "1"), s(:var_ref, s(:ident, "hello")))
-    node.type  #=> :if_mod
-    node[0]    #=> s(:int, "1")
-    node[0][0] #=> "1"
-
-(Note the `s()` syntax is shorthand for `AstNode.new(...)`. `s()` with no type
-is shorthand for a node of type `:list`)
-
-As shown, not all of the elements are AstNodes in themselves, some are String
-objects containing values. A list of only the AstNodes within a node can be
-accessed via the {YARD::Parser::Ruby::AstNode#children #children} method. Using
-the sexp declared above, we can do:
-
-    node.children #=> [s(:int, "1"), s(:var_ref, s(:ident, "hello"))]
-
-### AstNode#source and #line
-
-Every node defines the `#source` method which returns the source code that the
-node represents. One of the most common things to do with a node is to grab its
-source. The following example shows how this can be done:
-
-    source = "if 1 == 1 then\n  raise Exception\n end"
-    ast = YARD::Parser::Ruby::RubyParser.parse(source).root
-    ast[0].condition.source  #=> "1 == 1"
-    ast[0].then_block.source #=> "raise Exception"
-
-Note that this only works on source parsed from the RubyParser, not sexps
-declared using the `s()` syntax. This is because no source code is generated
-or stored by nodes. Instead, only the character ranges are stored, which are
-then looked up in the original full source string object. For example:
-
-    # Following the code snippet above
-    ast[0].then_block.source_range #=> 17..31
-
-We can also get the line and line ranges in a similar fashion:
-
-    ast[0].type       #=> :if
-    ast[0].line       #=> 1
-    ast[0].line_range #=> 1..3 (note the newlines in the source)
-
-### AstNode#jump
-
-Often the AST will be such that the node we care about might be buried arbitrarily
-deep in a node's hierarchy. The {YARD::Parser::Ruby::AstNode#jump} method exists
-to quickly get at a node of a specific type in such a situation:
-
-    # Get the first identifier in the statement
-    ast = s(s(:int, "1"), s(s(:var_ref, s(:ident, "hello"))))
-    ast.jump(:ident)[0] #=> "hello"
-
-Multiple types can be searched for at once. If none are found, the original root
-node is returned so that it may be chained.
-
-## The Legacy Parser
-
-The goal of the legacy parser is much the same as the new parser, but it is far
-more simplistic. Instead of a full-blown AST, the legacy parser simply groups
-together lists of "statements" called a {YARD::Parser::Ruby::Legacy::StatementList}.
-These statement lists are made up of {YARD::Parser::Ruby::Legacy::Statement} objects.
-A statement is any method call condition, loop, or declaration. Each statement
-may or may not have a block. In the case of a condition or loop, the block is
-the inner list of statements; in the case of a method call, the block is a do
-block (if provided). The statements themselves are made up of tokens, so instead
-of being semantic in nature like the new parser, statements are tied directly
-to the lexical tokens that make them up. To convert a statement into source, you
-simply join all the tokens together (this is done through the use of `#to_s`).
-
-Note that because there is little semantic parsing, the legacy parser is less
-able to deal with certain Ruby syntaxes. Specifically, the `:if_mod` syntax
-seen above ("hello if 1") would be considered two statements with the new parser,
-but using the legacy parser it is only one statement:
-
-    stmts = ARD::Parser::Ruby::Legacy::StatementList.new("hello if 1")
-    stmts[0].block       #=> nil
-    stmts[0].tokens.to_s #=> "hello if 1"
-
-In addition, this means that most handling still needs to be done via string
-manipulation and regular expression matching, making it considerably more
-difficult to use in edge case scenarios.