yard 0.7.5 → 0.8.0

data/docs/Handlers.md

@@ -27,8 +27,8 @@ state about what is being processed. For instance, the processor is what keeps
 track of the current namespace (the module or class an object is being defined
 in), scope (class or instance), file and owner. The owner refers to the object
 that is most directly responsible for the source statement being processed. This
-is most often the same as the namespace, except when parsing the body of a method, 
-where the namespace would be the class/module the method is defined in and the 
+is most often the same as the namespace, except when parsing the body of a method,
+where the namespace would be the class/module the method is defined in and the
 owner would be the method object itself.
 
 ## Implementing a Handler
@@ -42,13 +42,13 @@ class method. A very simple handler that handles a module definition would be:
 
     class MyModuleHandler < YARD::Handlers::Ruby::Base
       handles :module
-      
+
       def process
         puts "Handling a module named #{statement[0].source}"
       end
     end
-    
-For details on what nodes are, and what node types are, see the 
+
+For details on what nodes are, and what node types are, see the
 {file:docs/Parser.md parser architecture document}.
 
 In this case the node type being handled is the `:module` type. More than one
@@ -66,32 +66,32 @@ call can be handled by declaring the following in a `handles` statement:
 
     class MyHandler < YARD::Handlers::Ruby::Base
       handles method_call(:describe)
-      
+
       def process
         # Process the method call
       end
     end
-    
-In this case we handle any of the method calls to method name `describe` with 
+
+In this case we handle any of the method calls to method name `describe` with
 the following syntaxes:
-    
+
     describe(something)
     describe arg1, arg2, arg3
     describe(something) { perform_a_block }
     describe "Something" do
       a_block
     end
-    
+
 ### Creating a new Code Object
 
 Usually (but not always) handling is performed to create new code objects to add
 to the registry (for information about code objects, see {file:docs/CodeObjects.md this document}).
 Code objects should simply be created and added to the existing `namespace`. This
-will be enough to add them to the registry. There is also a convenience 
+will be enough to add them to the registry. There is also a convenience
 {YARD::Handlers::Base#register register} method which quickly sets standard attributed
 on the newly created object, such as the file, line, source and docstring of the
 object. This method will be seen in the next example.
-    
+
 ### Handling an Inner Block
 
 By default, the parser gives the processor class a list of all the top level
@@ -107,7 +107,7 @@ the following commands:
 
     class YARD::Handlers::Ruby::ModuleHandler < YARD::Handlers::Ruby::Base
       handles :module
-  
+
       def process
         modname = statement[0].source
         mod = register ModuleObject.new(namespace, modname)
@@ -126,27 +126,27 @@ Because the legacy handler uses the legacy parser and therefore a different kind
 of AST, there are subtle differences in the handler API. Most importantly, the
 `handles` method usually deals with either lexical tokens or source code as a string
 or RegExp object. The statement object, similarly, is made up of lexical tokens instead
-of semantically parsed nodes (this is described in the {file:docs/Parser.md parser document}). 
+of semantically parsed nodes (this is described in the {file:docs/Parser.md parser document}).
 
 The module example above can be rewritten as a legacy handler as follows:
 
     class YARD::Handlers::Ruby::Legacy::ModuleHandler < YARD::Handlers::Ruby::Legacy::Base
       handles TkMODULE
-  
+
       def process
         modname = statement.tokens.to_s[/^module\s+(#{NAMESPACEMATCH})/, 1]
         mod = register ModuleObject.new(namespace, modname)
         parse_block(:namespace => mod)
       end
     end
-    
+
 A few notes on the differences:
 
   * We inherit from `Legacy::Base` instead of the standard Ruby Base handler class.
   * We exchange node type `:module` for `TkMODULE`, which represents the
-    first token in the statement. 
-  * We perform direct string manipulation to get the module name. 
-  * `parse_block` does not take a list of statements. In the old parser API, 
-    each statement has a `block` attribute which defines the list of 
-    statements within that statement, if any. Therefore, `parse_block` will 
+    first token in the statement.
+  * We perform direct string manipulation to get the module name.
+  * `parse_block` does not take a list of statements. In the old parser API,
+    each statement has a `block` attribute which defines the list of
+    statements within that statement, if any. Therefore, `parse_block` will
     always parse the `statement.block` if it exists.

data/docs/Overview.md

@@ -11,14 +11,14 @@ that tools like RDoc do not do. These components are:
 * [Data Storage Component](#storage)
 * [Post Processing & Templating System](#templates)
 
-This separation is a major goal of the project, and means that YARD is not *just* 
-a tool to generate HTML output. The expectation is that any subset of YARD's 
-major components may be used, extended or modified independently. YARD may be 
-used just as a data gathering tool (to parse and audit code), just as a data 
-source (a webserver containing raw unformatted data about code), or just as a 
+This separation is a major goal of the project, and means that YARD is not *just*
+a tool to generate HTML output. The expectation is that any subset of YARD's
+major components may be used, extended or modified independently. YARD may be
+used just as a data gathering tool (to parse and audit code), just as a data
+source (a webserver containing raw unformatted data about code), or just as a
 conventional HTML documentation generation tool (like RDoc).
 
-The important classes and dependencies of these components are shown in the 
+The important classes and dependencies of these components are shown in the
 following class diagram:
 
 ![Overview Class Diagram](images/overview-class-diagram.png)
@@ -37,7 +37,7 @@ tasks during the data gathering process (*note: the tag architecture is not*
   * {file:docs/Tags.md}
 
 The parser component reads source files and converts it into a set of statements
-which the handlers then process, creating code objects which in turn create tags 
+which the handlers then process, creating code objects which in turn create tags
 (meta-data) attached to the objects. These objects are all added to the {YARD::Registry},
 the data store component.
 

data/docs/Parser.md

@@ -7,7 +7,7 @@ that runs before any handling is done on the source. The parser is meant to tran
 the source into a set of statements that can be understood by the {file:docs/Handlers.md Handlers}
 that run immediately afterwards.
 
-The important classes are described in the class diagram of the entire parser 
+The important classes are described in the class diagram of the entire parser
 system below:
 
 ![Parser Class Diagram](images/parser-class-diagram.png)
@@ -23,7 +23,7 @@ to parsing methods). YARD supports Ruby and C source files, but custom parsers c
 be implemented and registered for various other languages by subclassing `Parser::Base`
 and registering the parser with {YARD::Parser::SourceParser.register_parser_type}.
 
-This factory class should always be used when parsing source files rather than 
+This factory class should always be used when parsing source files rather than
 the individual parser classes since it initiates the pipeline that runs the
 handlers on the parsed source. The parser used must also match the handlers,
 and this is coordinated by the `SourceParser` class as well.
@@ -36,36 +36,36 @@ array of file globs or a single file glob.
 
     YARD::Parser::SourceParser.parse('spec_*.rb')
     YARD::Parser::SourceParser.parse(['spec_*.rb', '*_helper.rb'])
-    
+
 This is equivalent to the convenience method {YARD.parse}:
 
     YARD.parse('lib/**/*.rb')
-    
+
 In some cases (ie. for testing), it may be more helpful to parse a string of input
 directly. In such a case, the method {YARD::Parser::SourceParser.parse_string} should be
 used:
 
     YARD::Parser::SourceParser.parse_string("def method(a, b) end")
-    
+
 You can also provide the parser type explicitly as the second argument:
 
     # Parses a string of C
     YARD::Parser::SourceParser.parse_string("int main() { }", :c)
-    
-Note that these two methods are aliased as {YARD.parse} and {YARD.parse_string} for 
+
+Note that these two methods are aliased as {YARD.parse} and {YARD.parse_string} for
 convenience.
-    
+
 ## Implementing and Registering a Custom Parser
 
 To implement a custom parser, subclass {YARD::Parser::Base}. Documentation on which
 abstract methods should be implemented are documented in that class. After the class
 is implemented, it is registered with the {YARD::Parser::SourceParser} factory class
 to be called when a file of the right extension needs to be parsed, or when a user
-selects that parser type explicitly. To register your new parser class, call the 
+selects that parser type explicitly. To register your new parser class, call the
 method {YARD::Parser::SourceParser.register_parser_type}:
 
     SourceParser.register_parser_type(:my_parser, MyParser, 'my_parser_ext')
-    
+
 The last argument can be a single extension, a list of extensions (Array), a single Regexp, or a
 list of Regexps. Do not include the '.' in the extension.
 
@@ -73,11 +73,11 @@ list of Regexps. Do not include the '.' in the extension.
 ## The Two Ruby Parser Types
 
 When parsing Ruby, the SourceParser can either instantiate the new {YARD::Parser::Ruby::RubyParser}
-class or the {YARD::Parser::Ruby::Legacy::StatementList} class. The first of the 
-two, although faster, more robust and more efficient, is only available for 
-Ruby 1.9. The legacy parser parser is available in both 1.8.x and 1.9, if 
-compatibility is required. The choice of parser will affect which handlers 
-ultimately get used, since new handlers can only use the new parser and the 
+class or the {YARD::Parser::Ruby::Legacy::StatementList} class. The first of the
+two, although faster, more robust and more efficient, is only available for
+Ruby 1.9. The legacy parser parser is available in both 1.8.x and 1.9, if
+compatibility is required. The choice of parser will affect which handlers
+ultimately get used, since new handlers can only use the new parser and the
 same requirement applies to the legacy parser & handlers.
 
 ## Switching to Legacy Parser
@@ -89,7 +89,7 @@ only the legacy handlers are implemented, the `SourceParser` class should force
 the use of the legacy parser by setting the `parser_type` attribute as such:
 
     YARD::Parser::SourceParser.parser_type = :ruby18
-    
+
 The default value is `:ruby`. Note that this cannot be forced the other way around,
 a parser type of `:ruby` cannot be set under Ruby 1.8.x as the new parser is not
 supported under 1.8.
@@ -114,12 +114,12 @@ by the `#type` method. The following examples show some of the basic uses of `As
     node.type  #=> :if_mod
     node[0]    #=> s(:int, "1")
     node[0][0] #=> "1"
-    
+
 (Note the `s()` syntax is shorthand for `AstNode.new(...)`. `s()` with no type
 is shorthand for a node of type `:list`)
 
-As shown, not all of the elements are AstNodes in themselves, some are String 
-objects containing values. A list of only the AstNodes within a node can be 
+As shown, not all of the elements are AstNodes in themselves, some are String
+objects containing values. A list of only the AstNodes within a node can be
 accessed via the {YARD::Parser::Ruby::AstNode#children #children} method. Using
 the sexp declared above, we can do:
 
@@ -127,23 +127,23 @@ the sexp declared above, we can do:
 
 ### AstNode#source and #line
 
-Every node defines the `#source` method which returns the source code that the 
-node represents. One of the most common things to do with a node is to grab its 
+Every node defines the `#source` method which returns the source code that the
+node represents. One of the most common things to do with a node is to grab its
 source. The following example shows how this can be done:
 
     source = "if 1 == 1 then\n  raise Exception\n end"
     ast = YARD::Parser::Ruby::RubyParser.parse(source).root
     ast[0].condition.source  #=> "1 == 1"
     ast[0].then_block.source #=> "raise Exception"
-    
+
 Note that this only works on source parsed from the RubyParser, not sexps
 declared using the `s()` syntax. This is because no source code is generated
 or stored by nodes. Instead, only the character ranges are stored, which are
 then looked up in the original full source string object. For example:
 
     # Following the code snippet above
-    ast[0].then_block.source_range #=> 17..31 
-    
+    ast[0].then_block.source_range #=> 17..31
+
 We can also get the line and line ranges in a similar fashion:
 
     ast[0].type       #=> :if
@@ -159,7 +159,7 @@ to quickly get at a node of a specific type in such a situation:
     # Get the first identifier in the statement
     ast = s(s(:int, "1"), s(s(:var_ref, s(:ident, "hello"))))
     ast.jump(:ident)[0] #=> "hello"
-    
+
 Multiple types can be searched for at once. If none are found, the original root
 node is returned so that it may be chained.
 
@@ -167,11 +167,11 @@ node is returned so that it may be chained.
 
 The goal of the legacy parser is much the same as the new parser, but it is far
 more simplistic. Instead of a full-blown AST, the legacy parser simply groups
-together lists of "statements" called a {YARD::Parser::Ruby::Legacy::StatementList}. 
+together lists of "statements" called a {YARD::Parser::Ruby::Legacy::StatementList}.
 These statement lists are made up of {YARD::Parser::Ruby::Legacy::Statement} objects.
-A statement is any method call condition, loop, or declaration. Each statement 
-may or may not have a block. In the case of a condition or loop, the block is 
-the inner list of statements; in the case of a method call, the block is a do 
+A statement is any method call condition, loop, or declaration. Each statement
+may or may not have a block. In the case of a condition or loop, the block is
+the inner list of statements; in the case of a method call, the block is a do
 block (if provided). The statements themselves are made up of tokens, so instead
 of being semantic in nature like the new parser, statements are tied directly
 to the lexical tokens that make them up. To convert a statement into source, you
@@ -185,7 +185,7 @@ but using the legacy parser it is only one statement:
     stmts = ARD::Parser::Ruby::Legacy::StatementList.new("hello if 1")
     stmts[0].block       #=> nil
     stmts[0].tokens.to_s #=> "hello if 1"
-    
+
 In addition, this means that most handling still needs to be done via string
 manipulation and regular expression matching, making it considerably more
 difficult to use in edge case scenarios.

data/docs/Tags.md

@@ -2,585 +2,281 @@
 
 # Tags Overview
 
-Tags represent the metadata that can be added to documentation through the `@tag`
-style syntax:
-
-    # @tagname some data
-    class Foo
-    end
-
-The above example adds metadata under the name `tagname` to the Foo class object.
-
-Tags are the best way to add arbitrary metadata when documenting an object in a
-way to access it later without having to parse the entire comment string. The
-rest of the document will describe the tag syntax, how to access the tag 
-metadata and how to extend YARD to support custom tags or override existing tags.
+Tags represent meta-data as well as behavioural data that can be added to
+documentation through the `@tag` style syntax. As mentioned, there are two
+basic types of tags in YARD, "meta-data tags" and "behavioural tags", the
+latter is more often known as "directives". These two tag types can be
+visually identified by their prefix. Meta-data tags have a `@` prefix,
+while directives have a prefix of `@!` to indicate that the directive
+performs some potentially mutable action on or with the docstring. The
+two tag types would be used in the following way, respectively:
+
+    # @meta_data_tag some data
+    # @!directive_tag some data
+    class Foo; end
+
+This document describes how tags can be specified, how they affect your
+documentation, and how to use specific built-in tags in YARD, as well
+as how to define custom tags.
+
+## Meta-Data Tags
+
+Meta-data tags are useful to add arbitrary meta-data about a documented
+object. These tags simply add data to objects that can be looked up later,
+either programmatically, or displayed in templates. The benefit to describing
+objects using meta-data tags is that your documentation can be organized
+semantically. Rather than having a huge listing of text with no distinction
+of what each paragraph is discussing, tags allow you to focus in on specific
+elements of your documentation.
+
+For example, describing parameters of a method can often be important to your
+documentation, but should not be mixed up with the documentation that describes
+what the method itself does. In this case, separating the parameter documentation
+into {tag:param} tags can yield much better organized documentation, both in
+source and in your output, without having to manually format the data using
+standard markup.
+
+All of this meta-data can be easily parsed by tools and used both in your templates
+as well as in code checker tools. An example of how you can leverage tags
+programmatically is shown in the {tag:todo} tag, which lists a small snippet of
+Ruby code that can list all of your TODO items, if they are properly tagged.
+
+Custom meta-data tags can be added either programmatically or via the YARD
+command-line. This is discussed in the "[Adding Custom Tags](#Adding_Custom_Tags)"
+section.
+
+A list of built-in meta-data tags are found below in the [Tag List](#Tag_List).
+
+## Directives
+
+Directives are similar to meta-data tags in the way they are specified, but they
+do not add meta-data to the object directly. Instead, they affect the parsing
+context and objects themselves, allowing a developer to create objects
+(like methods) outright, rather than simply add text to an existing object.
+Directives have a `@!` prefix to differentiate these tags from meta-data tags,
+as well as to indicate that the tag may modify or create new objects when
+it is called.
+
+A list of built-in directives are found below in the [Directive List](#Directive_List).
 
 ## Tag Syntax
 
-Tags begin with "@tagname" at the start of a comment line. Tags can span multiple
-lines if the subsequent lines are indented by more than one space. The following
-syntax is valid:
+Tags begin with the `@` or `@!` prefix at the start of a comment line, followed
+immediately by the tag name, and then optional tag data (if the tag requires it).
+Unless otherwise specified by documentation for the tag, all "description" text
+is considered free-form data and can include any arbitrary textual data.
 
-    # @tagname This is 
-    #   tag data
-    # but this is not
-    
-In the above example, "@tagname" will have the text *"This is tag data"*.
+### Multi-line Tags
 
-If a tag's data begins with `(see NAME)` it is considered a "reference tag".
-The syntax and semantics of a reference tag are discussed in the section below
-titled "[Reference Tags](#reftags)"
+Tags can span multiple lines if the subsequent lines are indented by more than
+one space. The typical convention is to indent subsequent lines by 2 spaces.
+In the following example, `@tagname` will have the text *"This is indented tag data"*:
 
-Although custom tags can be parsed in any way, the built-in tags follow a few 
-common syntax structures by convention in order to simplify the syntax. The 
-following syntaxes are available:
+    # @tagname This is
+    #   indented tag data
+    # but this is not
 
-### Freeform Data
+For most tags, newlines and indented data are not significant and do not impact
+the result of the tag. In other words, you can decide to span a tag onto multiple
+lines at any point by creating an indented block. However, some tags like
+{tag:example}, {tag:overload}, {tag:!macro}, {tag:!method}, and {tag:!attribute}
+rely on the first line for special information about the tag, and you cannot
+split this first line up. For instance, the {tag:example} tag uses the first line
+to indicate the example's title.
 
-This syntax has no special syntax, it is simply a tagname followed by any
-data.
+### Common Tag Syntaxes
 
-    @tagname data here
-    
-### Freeform Data With Title
-
-Occasionally a freeform tag may reserve the first line for a title (or some
-other associative identifier) and treat only the subsequent indented lines as
-the tag data. Two examples are the `@example` and `@overload` tags. In the case
-of `@example` the first line is a title, and in the case of `@overload` the
-first line is the method signature for the overload. Here is an example of both:
-
-    @example Reverse a string
-      "hello world".reverse
-      
-    @overload request(method = :get, url = 'http://example.com')
-      Performs a request on +url+
-      @param [Symbol] method the request method
-      @param [String] url the URL to perform the request on
-      @return [String] the result body (no headers)
-
-### Data With Optional Type Information
-
-This syntax optionally contains type information to be associated with the
-tag. Type information is specified as a freeform list of Ruby types, duck
-types or literal values. The following is a valid tag with type information:
-
-    @return [String, #read] a string or object that responds to #read
-    
-### Data With Name and Optional Type Information
-
-A special case of the above data with optional type information is the case
-of tags like `@param`, where the data is further associated with a key. In
-the case of `@param` the key is an argument name in the method. The following
-shows how this can be used:
-
-    @param [String] url the URL to perform the request on
-
-Note that "url" in the above example is the key name. The syntax is of the form:
-
-    @tagname [types] <name> <description>
-    
-As mentioned, types are optional, so the following is also valid:
-
-    @param url the URL to perform the request on
-
-
-<a name="taglist"></a>
-
-## List of Available Tags
-
-YARD supplies the following built-in tags:
-
-  * `@abstract`: Marks a class/module/method as abstract with optional
-    implementor information.
-  
-        @abstract Subclass and override {#run} to implement a custom Threadable class.
-        
-  * `@api`: Declares the API that the object belongs to. Does not display in
-    output, but useful for performing queries (`yardoc --query`). Any text is
-    allowable in this tag, and there are no predefined values(*).
-    
-        @api freeform text
-        
-    (*) Note that the special name `@api private` does display a notice in 
-    documentation if it is listed, letting users know that the method is not
-    to be used.
-      
-  * `@attr`: Declares an attribute from the docstring of a class. Meant to be
-    used on Struct classes only (classes that inherit Struct).
-  
-        @attr [Types] attribute_name a full description of the attribute
-      
-  * `@attr_reader`: Declares a readonly attribute from the docstring of a class.
-    Meant to be used on Struct classes only (classes that inherit Struct). See `@attr`.
-  
-        @attr_reader [Types] name description of a readonly attribute
-      
-  * `@attr_writer`: Declares a writeonly attribute from the docstring of class.
-    Meant to be used on Struct classes only (classes that inherit Struct). See `@attr`.
-  
-        @attr_writer [Types] name description of writeonly attribute
-
-  * `@attribute`: Recognizes a DSL class method as an attribute with the given
-    name. Also accepts the r, w, or rw flag to signify that the attribute is 
-    readonly, writeonly, or readwrite (default). Only used with DSL methods.
-
-        @attribute [rw|r|w] NAME
-
-  * `@author`: List the author(s) of a class/method
-
-        @author Full Name
-
-  * `@deprecated`: Marks a method/class as deprecated with an optional
-    reason.
-
-        @deprecated Describe the reason or provide alt. references here
-
-  * `@example`: Show an example snippet of code for an object. The
-    first line is an optional title.
-
-        @example Reverse a string
-          "mystring".reverse #=> "gnirtsym"
-          
-  * `@macro`: Registers or expands a new macro. See the [Macros](#macros)
-    section for more details. Note that the name parameter is never optional.
-    
-        @macro [new|attached] macro_name
-          The macro contents to expand
-          
-  * `@method`: Recognizes a DSL class method as a method with the given name
-    and optional signature. Only used with DSL methods.
-      
-        @method method_signature(opts = {}, &block)
-  
-  * `@note`: Creates an emphasized note for the users to read about the
-    object.
-    
-        @note This method should only be used in outer space.
-
-  * `@option`: Describe an options hash in a method. The tag takes the
-    name of the options parameter first, followed by optional types,
-    the option key name, an optional default value for the key and a 
-    description of the option.
-
-        # @param [Hash] opts the options to create a message with.
-        # @option opts [String] :subject The subject
-        # @option opts [String] :from ('nobody') From address
-        # @option opts [String] :to Recipient email
-        # @option opts [String] :body ('') The email's body 
-        def send_email(opts = {})
-        end 
-
-  * `@overload`: Describe that your method can be used in various
-    contexts with various parameters or return types. The first
-    line should declare the new method signature, and the following
-    indented tag data will be a new documentation string with its
-    own tags adding metadata for such an overload.
-
-        # @overload set(key, value)
-        #   Sets a value on key
-        #   @param [Symbol] key describe key param
-        #   @param [Object] value describe value param
-        # @overload set(value)
-        #   Sets a value on the default key `:foo`
-        #   @param [Object] value describe value param
-        def set(*args)
-        end
-      
-  * `@param`: Defines method parameters
-
-        @param [optional, types, ...] argname description
-      
-  * `@private`: Defines an object as private. This exists for classes,
-    modules and constants that do not obey Ruby's visibility rules. For
-    instance, an inner class might be considered "private", though Ruby
-    would make no such distinction. By declaring the @private tag, the
-    class can be hidden from documentation by using the `--no-private`
-    command-line switch to yardoc (see {file:README.md}).
-  
-        @private
-        
-  * `@raise`: Describes an Exception that a method may throw
-
-        @raise [ExceptionClass] description
-
-  * `@return`: Describes return value of method
-
-        @return [optional, types, ...] description
-  
-  * `@scope`: Sets the scope of a DSL method. Only applicable to DSL method
-    calls. Acceptable values are 'class' or 'instance'
-    
-        @scope class|instance
-      
-  * `@see`: "See Also" references for an object. Accepts URLs or
-    other code objects with an optional description at the end.
-
-        @see http://example.com Description of URL
-        @see SomeOtherClass#method
-      
-  * `@since`: Lists the version the feature/object was first added
-
-        @since 1.2.4
-      
-  * `@todo`: Marks a TODO note in the object being documented
-
-        @todo Add support for Jabberwocky service
-          There is an open source Jabberwocky library available 
-          at http://somesite.com that can be integrated easily
-          into the project.
-
-  * `@version`: Lists the version of a class, module or method
-
-        @version 1.0
-
-  * `@visibility`: Sets the visibility of a DSL method. Only applicable to
-    DSL method calls. Acceptable values are public, protected, or private.
-    
-        @visibility public|protected|private
-
-  * `@yield`: Describes the block. Use types to list the parameter
-    names the block yields.
-
-        # for block {|a, b, c| ... }
-        @yield [a, b, c] Description of block
-
-  * `@yieldparam`: Defines parameters yielded by a block
-
-        @yieldparam [optional, types, ...] argname description
-
-  * `@yieldreturn`: Defines return type of a block
-
-        @yieldreturn [optional, types, ...] description
+Although custom tags can be parsed in any way, the built-in tags follow a few
+common syntax structures by convention in order to simplify the syntax. The
+following syntaxes are available:
 
+1. **Freeform data** &mdash; In this case, any amount of textual data is allowed,
+  including no data. In some cases, no data is necessary for the tag.
+2. **Freeform data with a types specifier list** &mdash; Mostly freeform data
+  beginning with an *optional* types specifier list surrounded in `[brackets]`.
+  Note that for extensibility, other bracket types are allowed, such as `<>`,
+  `()` and `{}`. The contents of the list are discussed in detail below.
+3. **Freeform data with a name and types specifier list** &mdash; freeform
+  data beginning with an *optional* types list, as well as a name key. The
+  name key is *required*. Note that for extensibility, the name can be placed
+  *before* the types list, like: `name [Types] description`. In this case,
+  a separating space is not required between the name and types, and you
+  can still use any of the other brackets that the type specifier list allows.
+4. **Freeform data with title** &mdash; freeform data where the first line cannot
+  be split into multiple lines. The first line must also always refer to the
+  "title" portion, and therefore, if there is no title, the first line must
+  be blank. The "title" might occasionally be listed by another name in tag
+  documentation, however, you can identify this syntax by the existence of
+  a multi-line signature with "Indented block" on the second line.
+
+In the tag list below, the term "description" implies freeform data, `[Types]`
+implies a types specifier list, "name" implies a name key, and "title" implies
+the first line is a newline significant field that cannot be split into multiple
+lines.
+
+### Types Specifier List
+
+In some cases, a tag will allow for a "types specifier list"; this will be evident
+from the use of the `[Types]` syntax in the tag signature. A types specifier list
+is a comma separated list of types, most often classes or modules, but occasionally
+literals. For example, the following {tag:return} tag lists a set of types returned
+by a method:
+
+    # Finds an object or list of objects in the db using a query
+    # @return [String, Array<String>, nil] the object or objects to
+    #   find in the database. Can be nil.
+    def find(query) finder_code_here end
+
+A list of conventions for type names is specified below. Typically, however,
+any Ruby literal or class/module is allowed here. Duck-types (method names
+prefixed with "#") are also allowed.
+
+Note that the type specifier list is always an optional field and can be omitted
+when present in a tag signature. This is the reason why it is surrounded by
+brackets. It is also a freeform list, and can contain any list of values, though
+a set of conventions for how to list types is described below.
+
+### Type List Conventions
+
+<p class="note">
+  A list of examples of common type listings and what they translate into is
+  available at <a href="http://yardoc.org/types">http://yardoc.org/types</a>.
+</p>
+
+Typically, a type list contains a list of classes or modules that are associated
+with the tag. In some cases, however, certain special values are allowed or required
+to be listed. This section discusses the syntax for specifying Ruby types inside of
+type specifier lists, as well as the other non-Ruby types that are accepted by
+convention in these lists.
+
+It's important to realize that the conventions listed here may not always adequately
+describe every type signature, and is not meant to be a complete syntax. This is
+why the types specifier list is freeform and can contain any set of values. The
+conventions defined here are only conventions, and if they do not work for your
+type specifications, you can define your own appropriate conventions.
+
+Note that a types specifier list might also be used for non-Type values. In this
+case, the tag documentation will describe what values are allowed within the
+type specifier list.
+
+#### Class or Module Types
+
+Any Ruby type is allowed as a class or module type. Such a type is simply the name
+of the class or module.
+
+Note that one extra type that is accepted by convention is the `Boolean` type,
+which represents both the `TrueClass` and `FalseClass` types. This type does not
+exist in Ruby, however.
+
+#### Parametrized Types
+
+In addition to basic types (like String or Array), YARD conventions allow for
+a "generics" like syntax to specify container objects or other parametrized types.
+The syntax is `Type<SubType, OtherSubType, ...>`. For instance, an Array might
+contain only String objects, in which case the type specification would be
+`Array<String>`. Multiple parametrized types can be listed, separated by commas.
+
+Note that parametrized types are typically not order-dependent, in other words,
+a list of parametrized types can occur in any order inside of a type. An array
+specified as `Array<String, Fixnum>` can contain any amount of Strings or Fixnums,
+in any order. When the order matters, use "order-dependent lists", described below.
+
+#### Duck-Types
+
+Duck-types are allowed in type specifier lists, and are identified by method
+names beginning with the "#" prefix. Typically, duck-types are recommended
+for {tag:param} tags only, though they can be used in other tags if needed.
+The following example shows a method that takes a parameter of any type
+that responds to the "read" method:
+
+    # Reads from any I/O object.
+    # @param [#read] io the input object to read from
+    def read(io) io.read end
+
+#### Hashes
+
+Hashes can be specified either via the parametrized type discussed above,
+in the form `Hash<KeyType, ValueType>`, or using the hash specific syntax:
+`Hash{KeyTypes=>ValueTypes}`. In the latter case, KeyTypes or ValueTypes can
+also be a list of types separated by commas.
+
+#### Order-Dependent Lists
+
+An order dependent list is a set of types surrounded by "()" and separated by
+commas. This list must contain exactly those types in exactly the order specified.
+For instance, an Array containing a String, Fixnum and Hash in that order (and
+having exactly those 3 elements) would be listed as: `Array<(String, Fixnum, Hash)>`.
+
+#### Literals
+
+Some literals are accepted by virtue of being Ruby literals, but also by YARD
+conventions. Here is a non-exhaustive list of certain accepted literal values:
+
+* `true`, `false`, `nil` &mdash; used when a method returns these explicit literal
+  values. Note that if your method returns both `true` or `false`, you should use
+  the `Boolean` conventional type instead.
+* `self` &mdash; has the same meaning as Ruby's "self" keyword in the context of
+  parameters or return types. Recommended mostly for {tag:return} tags that are
+  chainable.
+* `void` &mdash; indicates that the type for this tag is explicitly undefined.
+  Mostly used to specify {tag:return} tags that do not care about their return
+  value. Using a `void` return tag is recommended over no type, because it makes
+  the documentation more explicit about what the user should expect. YARD will
+  also add a note for the user if they have undefined return types, making things
+  clear that they should not use the return value of such a method.
 
 <a name="reftags"></a>
 
-## Reference Tags
-
-Although attempt is made in YARD to leave as many of the syntax details as
-possible to the factory provider, there is a special tag syntax for referencing
-tags created in other objects so that they can be reused again. This is common
-when an object describes a return type or parameters that are passed through to
-other methods. In such a case, it is more manageable to use the reference tag
-syntax. Consider the following example:
-
-    class User
-      # @param [String] username the nam of the user to add
-      # @param [Number] uid the user ID
-      # @param [Number] gid the group ID
-      def initialize(username, uid, gid)
-      end
-    end
-
-    module UserHelper
-      # @param (see User#initialize)
-      def add_user(username, uid, gid)
-        User.new(username, uid, gid)
-      end
-
-      # @param username (see User#initialize)
-      def add_root_user(username)
-        User.new(username, 0, 0)
-      end
-    end
-
-Because the UserHelper module methods delegate directly to `User.new`, copying
-the documentation details would be unmaintainable. In this case, the (see METHODNAME)
-syntax is used to reference the tags from the User constructor to the helper methods.
-For the first method, all `@param` tags are referenced in one shot, but the second
-method only references one of the tags by adding `username` before the reference.
-
-Reference tags are represented by the {YARD::Tags::RefTag} class and are created
-directly during parsing by {YARD::Docstring}.
-
-<a name="macros"></a>
-
-## Macros
-
-Macros enable the documenter to write repetitive documentation once and then
-re-apply it to other objects. Macros are defined on docstrings using the
-`@macro` tag. The same `@macro` tag is used to expand them. The following
-is an example of a simple macro declaration and expansion:
-
-    # @macro [new] string_attr
-    # @return [String] the attribute +$1+ as a String
-    attr_accessor :foo
-    
-    # @macro string_attr
-    attr_accessor :bar
-    
-In the above example, both attributes `foo` and `bar` will get the docstring
-that includes a return tag "the attribute as a String". It would be equivalent
-to writing the following:
-
-    # @return [String] the attribute +foo+ as a String
-    attr_accessor :foo
-
-    # @return [String] the attribute +bar+ as a String
-    attr_accessor :bar
-
-### Creating a Macro
-
-If the macro does not already exist, it will be created if:
-
-  1. there are interpolation variables (`$1`, `$2`, `${3-5}`, etc.) in the
-     docstring, or,
-  2. the macro is specified with the `[new]` or `[attached]` flag.
-
-For instance, creating a new macro might look like (see the section on
-interpolation below for a description of the `$2` syntax):
-
-    # @macro the_macro_name
-    # @return [$2] the thing to return
-    typed_attribute :foo, String
-
-Or:
-
-    # @macro [new] the_macro_name
-    # Returns a string!
-    def foo; end
-    
-You can also "attach" a macro to a method if it is in the class scope. In
-this case, you do not need to also provide the 'new' flag, using 'attach'
-is sufficient:
-
-    # @macro [attach] the_macro_name
-    #   @return [String] the string value
-    def self.foo; end
-
-Any time 'foo' is called in the class scope of an inheriting class, the macro
-will automatically expand on that method call (potentially creating a new
-method object). Attaching macros is discussed below.
-
-Note that the name is never optional. Even if the macro is never re-used,
-it must be named.
-    
-### Indenting the Macro Data
-
-If a macro tag has an indented body of macro data (shown below), it will be
-the only portion of the docstring saved for re-use.
-
-    # @macro [new] macro_name
-    #   The macro data is here.
-    # This data is only used for the current object
-    def method; end
-    
-In the above case, "The macro data is here." is the only portion that will be
-re-used if the macro is called again on another object. However, for the case
-of the above method, both the macro data and the local docstring will be
-applied to the method, creating the docstring:
-
-    # The macro data is here.
-    # This data is only used for the current object.
-    def method; end
-    
-You can therefore keep portions of docstrings local to objects even when
-creating a macro, by indenting the portion of the data that should be re-
-expanded, and listing the local docstring data above or below.
-
-If there is no indented macro data, the entire docstring is saved as the
-macro data. For instance,
-
-    # @macro [new] macro_name
-    # The macro data is here.
-    # This data is also used for all macros.
-    def method; end
-    
-In the above case, the macro 'macro_name' will always show both lines of text
-when expanded on other objects.
-
-### Attaching a Macro to a DSL (Class) Method
-
-Macros can be created on class level methods (or class level method calls) in 
-order to implicitly expand a macro whenever that method is subsequently called 
-in a class, or any class that mixes in or inherits the method. These macros
-are called "attached" and are declared with the `[attach]` flag. For instance, 
-a library that uses a class level method call `property` in its codebase can 
-document these declarations in any future call like so:
-
-    class Resource
-      # Defines a new property
-      # @param [String] name the property name
-      # @param [Class] type the property's type
-      # @macro [attach] property
-      #   @return [$2] the $1 property
-      def self.property(name, type) end
-    end
-    
-    class Post < Resource
-      property :title, String
-      property :view_count, Integer
-    end
-
-If you cannot declare the macro on the actual method declaration, you can
-arbitrarily attach the macro to any method call. Suppose we only had the
-Post class in our codebase, we could add the macro to the first usage of
-the `property` call:
-
-    class Post < Resource
-      # @macro [attach] property
-      # @return [$2] the $1 property
-      property :title, String
-      property :view_count, Integer
-    end
-
-### Macro Variable Interpolation Syntax
-
-The interpolation syntax is similar to Ruby's regular expression variable syntax.
-It uses $1, $2, $3, ..., referring to the Nth argument in the method call. Using
-the above property example, $1 would be 'title', and $2 would be 'String'.
-$0 is a special variable that refers to the method call itself, in this case
-'property'. Finally, there is a $& variable which refers to the full line,
-or 'property :title, String'.
-
-#### Ranges
-
-Ranges are also acceptable with the syntax `${N-M}`. Negative values on either
-N or M are valid, and refer to indexes from the end of the list. Consider
-a DSL method that creates a method using the first argument with argument
-names following, ending with the return type of the method. This could be
-documented as:
-
-    # @macro dsl_method
-    # @method $1(${2--2})
-    # @return [${-1}] the return value of $0
-    create_method_with_args :foo, :a, :b, :c, String
-
-As described, the method is using the signature `foo(a, b, c)` and the return
-type from the last argument, `String`. When using ranges, tokens are joined
-with commas. Note that this includes using $0:
-
-    $0-1 # => Interpolates to "create_method_with_args, foo"
-
-If you want to separate them with spaces, use `$1 $2 $3 $4 ...`. Note that
-if the token cannot be expanded, it will return the empty string (not an error),
-so it would be safe to list `$1 $2 ... $10`, for example.
-
-#### Escaping Interpolation
-
-Interpolation can be escaped by prefixing the `$` with `\`, like so: 
-
-    # @macro foo
-    #   I have \$2.00 USD.
-
-## Programmatic API
-
-### Accessing Tag Information
-
-Tag metadata is added when a {YARD::Docstring} is added to a {file:docs/CodeObjects.md code object}
-using the {YARD::CodeObjects::Base#docstring=} attribute. In addition to adding
-conventional comments, tags are parsed and associated with the object. The easiest
-way to access tags on an object is to use the {YARD::CodeObjects::Base#tag} and `#tags`
-methods, for example:
-
-    # Using the Foo class object from above
-    obj.tags(:tagname).first.text #=> "some data"
-    
-Because multiple tags can be stored with the same name, they are stored as a list
-of tags. The `#tag` method is an alias for the first item in the list of tags.
-Also note that the `#tag`, `#tags` and `#has_tag?` methods are all convenience
-methods that delegate to the {YARD::Docstring} object described above.
-
-### Adding Custom Tags
-
-The `@tagname` tag used in the above examples is clearly not part of the tags
-that come with YARD. If such a tag would actually be part of documentation under
-a default install, YARD would raise a warning that the tag does not exist. It is,
-however, trivial to add this tag to be recognized by YARD.
-
-All tags in YARD are added to the {YARD::Tags::Library tag library} which makes
-use of a tag factory class to parse the data inside the tags. To simply add a
-tag that stores simple text like our `@tagname` tag above, use:
-
-    YARD::Tags::Library.define_tag("A Sample Tag", :tagname)
-
-This will now allow YARD to add the metadata from `@tagname` to the docstring.
-
-## Tag Factory Architecture
-
-Recognizing a tag is one part of the process. Parsing the tag contents is the
-second step. YARD has a tag architecture that allows developers to add or completely
-change the way tags contents can be parsed.
-
-The separation of registration and tag creation can be seen in the following
-class diagram:
-
-![Tags Architecture Class Diagram](images/tags-class-diagram.png)
-
-### DefaultFactory
-
-By default, YARD has a few standard syntaxes that can be parsed for tags. These
-are all implemented by the {YARD::Tags::DefaultFactory} class. These syntaxes
-are:
-
-  * Standard text: no parsing is done, but text is stripped of newlines and
-    multiple spaces.
-    
-  * Raw text: does no parsing at all, no stripping of newlines or spaces. This
-    is best used for code snippets.
+### Reference Tags
 
-  * Raw text with title: does no parsing on the text but extracts the first line
-    of the metadata as the "title", useful for tags such as `@example`:
-        
-        # @example Inspect an element
-        #   myobj.inspect #=> #<Object:0x123525>
+<p class="note">
+  Reference tag syntax applies only to meta-data tags, not directives.
+</p>
 
-  * Text with types: parses a list of types at the beginning of the text. Types
-    are optional. The standard syntax is in the form `[type1, type2, ...]`, 
-    for example:
-        
-        # @return [String, Symbol] a description here
-        # @return description here with no types
-    
-  * Text with types and a name: parses a list of types at the beginning of text
-    followed by a name and extra descriptive text. For example:
-        
-        # @param [String] str the string to reverse
-        def reverse(str) '...' end
+If a tag's data begins with `(see OBJECT)` it is considered a "reference tag".
+A reference tag literally copies the tag data by the given tag name from the
+specified OBJECT. For instance, a method may copy all {tag:param} tags from
+a given object using the reference tag syntax:
 
-As mentioned above, this syntax is implemented by the `DefaultFactory` which can
-be swapped out for any factory. In some cases, a developer may want to change
-the type declaration syntax to be in the form:
+    # @param [String] user the username for the operation
+    # @param [String] host the host that this user is associated with
+    # @param [Time] time the time that this operation took place
+    def clean(user, host, time = Time.now) end
 
-    # @tagname name <Types, here> description
-    
-This can be done by simply implementing a new factory that parses the data in
-this form.
+    # @param (see #clean)
+    def activate(user, host, time = Time.now) end
 
-### Implementing a Factory
+## Adding Custom Tags
 
-Factories should implement the method `parse_tag` as well as any `parse_tag_SUFFIX`
-method where SUFFIX refers to the suffix added when declaring the tag. For example,
-a tag can also be declared as follows:
+<p class="note">If a tag is specific to a given project, consider namespacing
+  it by naming it in the form <tt>projectname.tagname</tt>, ie.,
+  <tt>yard.tag_signature</tt>.</p>
 
-    YARD::Tags::Library.define_tag "Parameter", :param, :with_types
-    
-In such a case, the factory will be called with method `parse_tag_with_types`. In
-all cases, the method should return a new {YARD::Tags::Tag} object. Generally,
-the `parse_tag` methods take 2 or 3 parameters. A simple tag can be implemented
-as:
+Custom tags can be added to YARD either via the command-line or programmatically.
+The programmatic method is not discussed in this document, but rather in the
+{file:docs/TagsArch.md} document.
 
-    def parse_tag(tag_name, text)
-      Tag.new(tag_name, text)
-    end
+To add a custom tag via the command-line or .yardopts file, you can use the
+`--*-tag` options. A few different options are available for the common tag
+syntaxes described above. For example, to add a basic freeform tag, use:
 
-The text parameter contains pre-parsed text with extra spaces and newlines removed.
-If required, the method could also be declared with a third parameter containing
-unmodified raw text:
+    !!!sh
+    $ yard doc --tag rest_url:"REST URL"
 
-    def parse_tag_with_raw_text(tag_name, text, raw_text)
-      Tag.new(tag_name, raw_text)
-    end
+This will register the `@rest_url` tag for use in your documentation and display
+this tag in HTML output wherever it is used with the heading "REST URL".
+Note that the tag title should follow the tag name with a colon (`:`). Other
+tag syntaxes exist, such as the type specifier list freeform tag
+(`--type-tag`), or a named key tag with types (`--type-name-tag`).
 
-Note that this method would be invoked for a tag declared with the `:with_raw_text`
-suffix.
+If you want to create a tag but not display it in output (it is only for
+programmatic use), add `--hide-tag tagname` after the definition:
 
-### Changing the Factory
+    !!!sh
+    $ yard doc --tag complexity:"McCabe Complexity" --hide-tag complexity
 
-To change the factory, set the {YARD::Tags::Library.default_factory} attribute:
+Note that you might not need a tag title if you are hiding it. The title
+part can be omitted.
 
-    YARD::Tags::Library.default_factory = MyFactory
+{yard:include_tags}
 
-This must be done before any parsing is done, or the factory will not be used.