pygments.rb 1.2.1 → 2.0.0

data/vendor/pygments-main/doc/docs/api.rst

@@ -1,354 +0,0 @@
-.. -*- mode: rst -*-
-
-=====================
-The full Pygments API
-=====================
-
-This page describes the Pygments API.
-
-High-level API
-==============
-
-.. module:: pygments
-
-Functions from the :mod:`pygments` module:
-
-.. function:: lex(code, lexer)
-
-    Lex `code` with the `lexer` (must be a `Lexer` instance)
-    and return an iterable of tokens. Currently, this only calls
-    `lexer.get_tokens()`.
-
-.. function:: format(tokens, formatter, outfile=None)
-
-    Format a token stream (iterable of tokens) `tokens` with the
-    `formatter` (must be a `Formatter` instance). The result is
-    written to `outfile`, or if that is ``None``, returned as a
-    string.
-
-.. function:: highlight(code, lexer, formatter, outfile=None)
-
-    This is the most high-level highlighting function.
-    It combines `lex` and `format` in one function.
-
-
-.. module:: pygments.lexers
-
-Functions from :mod:`pygments.lexers`:
-
-.. function:: get_lexer_by_name(alias, **options)
-
-    Return an instance of a `Lexer` subclass that has `alias` in its
-    aliases list. The lexer is given the `options` at its
-    instantiation.
-
-    Will raise :exc:`pygments.util.ClassNotFound` if no lexer with that alias is
-    found.
-
-.. function:: get_lexer_for_filename(fn, **options)
-
-    Return a `Lexer` subclass instance that has a filename pattern
-    matching `fn`. The lexer is given the `options` at its
-    instantiation.
-
-    Will raise :exc:`pygments.util.ClassNotFound` if no lexer for that filename
-    is found.
-
-.. function:: get_lexer_for_mimetype(mime, **options)
-
-    Return a `Lexer` subclass instance that has `mime` in its mimetype
-    list. The lexer is given the `options` at its instantiation.
-
-    Will raise :exc:`pygments.util.ClassNotFound` if not lexer for that mimetype
-    is found.
-
-.. function:: load_lexer_from_file(filename, lexername="CustomLexer", **options)
-
-    Return a `Lexer` subclass instance loaded from the provided file, relative
-    to the current directory. The file is expected to contain a Lexer class
-    named `lexername` (by default, CustomLexer). Users should be very careful with
-    the input, because this method is equivalent to running eval on the input file.
-    The lexer is given the `options` at its instantiation.
-
-    :exc:`ClassNotFound` is raised if there are any errors loading the Lexer
-
-    .. versionadded:: 2.2
-
-.. function:: guess_lexer(text, **options)
-
-    Return a `Lexer` subclass instance that's guessed from the text in
-    `text`. For that, the :meth:`.analyse_text()` method of every known lexer
-    class is called with the text as argument, and the lexer which returned the
-    highest value will be instantiated and returned.
-
-    :exc:`pygments.util.ClassNotFound` is raised if no lexer thinks it can
-    handle the content.
-
-.. function:: guess_lexer_for_filename(filename, text, **options)
-
-    As :func:`guess_lexer()`, but only lexers which have a pattern in `filenames`
-    or `alias_filenames` that matches `filename` are taken into consideration.
-
-    :exc:`pygments.util.ClassNotFound` is raised if no lexer thinks it can
-    handle the content.
-
-.. function:: get_all_lexers()
-
-    Return an iterable over all registered lexers, yielding tuples in the
-    format::
-
-    	(longname, tuple of aliases, tuple of filename patterns, tuple of mimetypes)
-
-    .. versionadded:: 0.6
-
-.. function:: find_lexer_class_by_name(alias)
-
-    Return the `Lexer` subclass that has `alias` in its aliases list, without
-    instantiating it.
-
-    Will raise :exc:`pygments.util.ClassNotFound` if no lexer with that alias is
-    found.
-
-    .. versionadded:: 2.2
-
-.. function:: find_lexer_class(name)
-
-    Return the `Lexer` subclass that with the *name* attribute as given by
-    the *name* argument.
-
-
-.. module:: pygments.formatters
-
-Functions from :mod:`pygments.formatters`:
-
-.. function:: get_formatter_by_name(alias, **options)
-
-    Return an instance of a :class:`.Formatter` subclass that has `alias` in its
-    aliases list. The formatter is given the `options` at its instantiation.
-
-    Will raise :exc:`pygments.util.ClassNotFound` if no formatter with that
-    alias is found.
-
-.. function:: get_formatter_for_filename(fn, **options)
-
-    Return a :class:`.Formatter` subclass instance that has a filename pattern
-    matching `fn`. The formatter is given the `options` at its instantiation.
-
-    Will raise :exc:`pygments.util.ClassNotFound` if no formatter for that filename
-    is found.
-
-.. function:: load_formatter_from_file(filename, formattername="CustomFormatter", **options)
-
-    Return a `Formatter` subclass instance loaded from the provided file, relative
-    to the current directory. The file is expected to contain a Formatter class
-    named ``formattername`` (by default, CustomFormatter). Users should be very
-    careful with the input, because this method is equivalent to running eval
-    on the input file. The formatter is given the `options` at its instantiation.
-
-    :exc:`ClassNotFound` is raised if there are any errors loading the Formatter
-
-    .. versionadded:: 2.2
-
-.. module:: pygments.styles
-
-Functions from :mod:`pygments.styles`:
-
-.. function:: get_style_by_name(name)
-
-    Return a style class by its short name. The names of the builtin styles
-    are listed in :data:`pygments.styles.STYLE_MAP`.
-
-    Will raise :exc:`pygments.util.ClassNotFound` if no style of that name is
-    found.
-
-.. function:: get_all_styles()
-
-    Return an iterable over all registered styles, yielding their names.
-
-    .. versionadded:: 0.6
-
-
-.. module:: pygments.lexer
-
-Lexers
-======
-
-The base lexer class from which all lexers are derived is:
-
-.. class:: Lexer(**options)
-
-    The constructor takes a \*\*keywords dictionary of options.
-    Every subclass must first process its own options and then call
-    the `Lexer` constructor, since it processes the `stripnl`,
-    `stripall` and `tabsize` options.
-
-    An example looks like this:
-
-    .. sourcecode:: python
-
-        def __init__(self, **options):
-            self.compress = options.get('compress', '')
-            Lexer.__init__(self, **options)
-
-    As these options must all be specifiable as strings (due to the
-    command line usage), there are various utility functions
-    available to help with that, see `Option processing`_.
-
-    .. method:: get_tokens(text)
-
-        This method is the basic interface of a lexer. It is called by
-        the `highlight()` function. It must process the text and return an
-        iterable of ``(tokentype, value)`` pairs from `text`.
-
-        Normally, you don't need to override this method. The default
-        implementation processes the `stripnl`, `stripall` and `tabsize`
-        options and then yields all tokens from `get_tokens_unprocessed()`,
-        with the ``index`` dropped.
-
-    .. method:: get_tokens_unprocessed(text)
-
-        This method should process the text and return an iterable of
-        ``(index, tokentype, value)`` tuples where ``index`` is the starting
-        position of the token within the input text.
-
-        This method must be overridden by subclasses.
-
-    .. staticmethod:: analyse_text(text)
-
-        A static method which is called for lexer guessing. It should analyse
-        the text and return a float in the range from ``0.0`` to ``1.0``.
-        If it returns ``0.0``, the lexer will not be selected as the most
-        probable one, if it returns ``1.0``, it will be selected immediately.
-
-        .. note:: You don't have to add ``@staticmethod`` to the definition of
-                  this method, this will be taken care of by the Lexer's metaclass.
-
-    For a list of known tokens have a look at the :doc:`tokens` page.
-
-    A lexer also can have the following attributes (in fact, they are mandatory
-    except `alias_filenames`) that are used by the builtin lookup mechanism.
-
-    .. attribute:: name
-
-        Full name for the lexer, in human-readable form.
-
-    .. attribute:: aliases
-
-        A list of short, unique identifiers that can be used to lookup
-        the lexer from a list, e.g. using `get_lexer_by_name()`.
-
-    .. attribute:: filenames
-
-        A list of `fnmatch` patterns that match filenames which contain
-        content for this lexer. The patterns in this list should be unique among
-        all lexers.
-
-    .. attribute:: alias_filenames
-
-        A list of `fnmatch` patterns that match filenames which may or may not
-        contain content for this lexer. This list is used by the
-        :func:`.guess_lexer_for_filename()` function, to determine which lexers
-        are then included in guessing the correct one. That means that
-        e.g. every lexer for HTML and a template language should include
-        ``\*.html`` in this list.
-
-    .. attribute:: mimetypes
-
-        A list of MIME types for content that can be lexed with this
-        lexer.
-
-
-.. module:: pygments.formatter
-
-Formatters
-==========
-
-A formatter is derived from this class:
-
-
-.. class:: Formatter(**options)
-
-    As with lexers, this constructor processes options and then must call the
-    base class :meth:`__init__`.
-
-    The :class:`Formatter` class recognizes the options `style`, `full` and
-    `title`.  It is up to the formatter class whether it uses them.
-
-    .. method:: get_style_defs(arg='')
-
-        This method must return statements or declarations suitable to define
-        the current style for subsequent highlighted text (e.g. CSS classes
-        in the `HTMLFormatter`).
-
-        The optional argument `arg` can be used to modify the generation and
-        is formatter dependent (it is standardized because it can be given on
-        the command line).
-
-        This method is called by the ``-S`` :doc:`command-line option <cmdline>`,
-        the `arg` is then given by the ``-a`` option.
-
-    .. method:: format(tokensource, outfile)
-
-        This method must format the tokens from the `tokensource` iterable and
-        write the formatted version to the file object `outfile`.
-
-        Formatter options can control how exactly the tokens are converted.
-
-    .. versionadded:: 0.7
-       A formatter must have the following attributes that are used by the
-       builtin lookup mechanism.
-
-    .. attribute:: name
-
-        Full name for the formatter, in human-readable form.
-
-    .. attribute:: aliases
-
-        A list of short, unique identifiers that can be used to lookup
-        the formatter from a list, e.g. using :func:`.get_formatter_by_name()`.
-
-    .. attribute:: filenames
-
-        A list of :mod:`fnmatch` patterns that match filenames for which this
-        formatter can produce output. The patterns in this list should be unique
-        among all formatters.
-
-
-.. module:: pygments.util
-
-Option processing
-=================
-
-The :mod:`pygments.util` module has some utility functions usable for option
-processing:
-
-.. exception:: OptionError
-
-    This exception will be raised by all option processing functions if
-    the type or value of the argument is not correct.
-
-.. function:: get_bool_opt(options, optname, default=None)
-
-    Interpret the key `optname` from the dictionary `options` as a boolean and
-    return it. Return `default` if `optname` is not in `options`.
-
-    The valid string values for ``True`` are ``1``, ``yes``, ``true`` and
-    ``on``, the ones for ``False`` are ``0``, ``no``, ``false`` and ``off``
-    (matched case-insensitively).
-
-.. function:: get_int_opt(options, optname, default=None)
-
-    As :func:`get_bool_opt`, but interpret the value as an integer.
-
-.. function:: get_list_opt(options, optname, default=None)
-
-    If the key `optname` from the dictionary `options` is a string,
-    split it at whitespace and return it. If it is already a list
-    or a tuple, it is returned as a list.
-
-.. function:: get_choice_opt(options, optname, allowed, default=None)
-
-    If the key `optname` from the dictionary is not in the sequence
-    `allowed`, raise an error, otherwise return it.
-
-    .. versionadded:: 0.8

data/vendor/pygments-main/doc/docs/authors.rst

@@ -1,4 +0,0 @@
-Full contributor list
-=====================
-
-.. include:: ../../AUTHORS

data/vendor/pygments-main/doc/docs/changelog.rst

@@ -1 +0,0 @@
-.. include:: ../../CHANGES

data/vendor/pygments-main/doc/docs/cmdline.rst

@@ -1,166 +0,0 @@
-.. -*- mode: rst -*-
-
-======================
-Command Line Interface
-======================
-
-You can use Pygments from the shell, provided you installed the
-:program:`pygmentize` script::
-
-    $ pygmentize test.py
-    print "Hello World"
-
-will print the file test.py to standard output, using the Python lexer
-(inferred from the file name extension) and the terminal formatter (because
-you didn't give an explicit formatter name).
-
-If you want HTML output::
-
-    $ pygmentize -f html -l python -o test.html test.py
-
-As you can see, the -l option explicitly selects a lexer. As seen above, if you
-give an input file name and it has an extension that Pygments recognizes, you can
-omit this option.
-
-The ``-o`` option gives an output file name. If it is not given, output is
-written to stdout.
-
-The ``-f`` option selects a formatter (as with ``-l``, it can also be omitted
-if an output file name is given and has a supported extension).
-If no output file name is given and ``-f`` is omitted, the
-:class:`.TerminalFormatter` is used.
-
-The above command could therefore also be given as::
-
-    $ pygmentize -o test.html test.py
-
-To create a full HTML document, including line numbers and stylesheet (using the
-"emacs" style), highlighting the Python file ``test.py`` to ``test.html``::
-
-    $ pygmentize -O full,style=emacs -o test.html test.py
-
-
-Options and filters
--------------------
-
-Lexer and formatter options can be given using the ``-O`` option::
-
-    $ pygmentize -f html -O style=colorful,linenos=1 -l python test.py
-
-Be sure to enclose the option string in quotes if it contains any special shell
-characters, such as spaces or expansion wildcards like ``*``. If an option
-expects a list value, separate the list entries with spaces (you'll have to
-quote the option value in this case too, so that the shell doesn't split it).
-
-Since the ``-O`` option argument is split at commas and expects the split values
-to be of the form ``name=value``, you can't give an option value that contains
-commas or equals signs.  Therefore, an option ``-P`` is provided (as of Pygments
-0.9) that works like ``-O`` but can only pass one option per ``-P``. Its value
-can then contain all characters::
-
-    $ pygmentize -P "heading=Pygments, the Python highlighter" ...
-
-Filters are added to the token stream using the ``-F`` option::
-
-    $ pygmentize -f html -l pascal -F keywordcase:case=upper main.pas
-
-As you see, options for the filter are given after a colon. As for ``-O``, the
-filter name and options must be one shell word, so there may not be any spaces
-around the colon.
-
-
-Generating styles
------------------
-
-Formatters normally don't output full style information.  For example, the HTML
-formatter by default only outputs ``<span>`` tags with ``class`` attributes.
-Therefore, there's a special ``-S`` option for generating style definitions.
-Usage is as follows::
-
-    $ pygmentize -f html -S colorful -a .syntax
-
-generates a CSS style sheet (because you selected the HTML formatter) for
-the "colorful" style prepending a ".syntax" selector to all style rules.
-
-For an explanation what ``-a`` means for :doc:`a particular formatter
-<formatters>`, look for the `arg` argument for the formatter's
-:meth:`.get_style_defs()` method.
-
-
-Getting lexer names
--------------------
-
-.. versionadded:: 1.0
-
-The ``-N`` option guesses a lexer name for a given filename, so that ::
-
-    $ pygmentize -N setup.py
-
-will print out ``python``.  It won't highlight anything yet.  If no specific
-lexer is known for that filename, ``text`` is printed.
-
-Custom Lexers and Formatters
-----------------------------
-
-.. versionadded:: 2.2
-
-The ``-x`` flag enables custom lexers and formatters to be loaded
-from files relative to the current directory. Create a file with a class named
-CustomLexer or CustomFormatter, then specify it on the command line::
-
-    $ pygmentize -l your_lexer.py -f your_formatter.py -x
-
-You can also specify the name of your class with a colon::
-
-    $ pygmentize -l your_lexer.py:SomeLexer -x
-
-For more information, see :doc:`the Pygments documentation on Lexer development
-<lexerdevelopment>`.
-
-Getting help
-------------
-
-The ``-L`` option lists lexers, formatters, along with their short
-names and supported file name extensions, styles and filters. If you want to see
-only one category, give it as an argument::
-
-    $ pygmentize -L filters
-
-will list only all installed filters.
-
-The ``-H`` option will give you detailed information (the same that can be found
-in this documentation) about a lexer, formatter or filter. Usage is as follows::
-
-    $ pygmentize -H formatter html
-
-will print the help for the HTML formatter, while ::
-
-    $ pygmentize -H lexer python
-
-will print the help for the Python lexer, etc.
-
-
-A note on encodings
--------------------
-
-.. versionadded:: 0.9
-
-Pygments tries to be smart regarding encodings in the formatting process:
-
-* If you give an ``encoding`` option, it will be used as the input and
-  output encoding.
-
-* If you give an ``outencoding`` option, it will override ``encoding``
-  as the output encoding.
-
-* If you give an ``inencoding`` option, it will override ``encoding``
-  as the input encoding.
-
-* If you don't give an encoding and have given an output file, the default
-  encoding for lexer and formatter is the terminal encoding or the default
-  locale encoding of the system.  As a last resort, ``latin1`` is used (which
-  will pass through all non-ASCII characters).
-
-* If you don't give an encoding and haven't given an output file (that means
-  output is written to the console), the default encoding for lexer and
-  formatter is the terminal encoding (``sys.stdout.encoding``).

data/vendor/pygments-main/doc/docs/filterdevelopment.rst

@@ -1,71 +0,0 @@
-.. -*- mode: rst -*-
-
-=====================
-Write your own filter
-=====================
-
-.. versionadded:: 0.7
-
-Writing own filters is very easy. All you have to do is to subclass
-the `Filter` class and override the `filter` method. Additionally a
-filter is instantiated with some keyword arguments you can use to
-adjust the behavior of your filter.
-
-
-Subclassing Filters
-===================
-
-As an example, we write a filter that converts all `Name.Function` tokens
-to normal `Name` tokens to make the output less colorful.
-
-.. sourcecode:: python
-
-    from pygments.util import get_bool_opt
-    from pygments.token import Name
-    from pygments.filter import Filter
-
-    class UncolorFilter(Filter):
-
-        def __init__(self, **options):
-            Filter.__init__(self, **options)
-            self.class_too = get_bool_opt(options, 'classtoo')
-
-        def filter(self, lexer, stream):
-            for ttype, value in stream:
-                if ttype is Name.Function or (self.class_too and
-                                              ttype is Name.Class):
-                    ttype = Name
-                yield ttype, value
-
-Some notes on the `lexer` argument: that can be quite confusing since it doesn't
-need to be a lexer instance. If a filter was added by using the `add_filter()`
-function of lexers, that lexer is registered for the filter. In that case
-`lexer` will refer to the lexer that has registered the filter. It *can* be used
-to access options passed to a lexer. Because it could be `None` you always have
-to check for that case if you access it.
-
-
-Using a decorator
-=================
-
-You can also use the `simplefilter` decorator from the `pygments.filter` module:
-
-.. sourcecode:: python
-
-    from pygments.util import get_bool_opt
-    from pygments.token import Name
-    from pygments.filter import simplefilter
-
-
-    @simplefilter
-    def uncolor(self, lexer, stream, options):
-        class_too = get_bool_opt(options, 'classtoo')
-        for ttype, value in stream:
-            if ttype is Name.Function or (class_too and
-                                          ttype is Name.Class):
-                ttype = Name
-            yield ttype, value
-
-The decorator automatically subclasses an internal filter class and uses the
-decorated function as a method for filtering.  (That's why there is a `self`
-argument that you probably won't end up using in the method.)

data/vendor/pygments-main/doc/docs/filters.rst

@@ -1,41 +0,0 @@
-.. -*- mode: rst -*-
-
-=======
-Filters
-=======
-
-.. versionadded:: 0.7
-
-You can filter token streams coming from lexers to improve or annotate the
-output. For example, you can highlight special words in comments, convert
-keywords to upper or lowercase to enforce a style guide etc.
-
-To apply a filter, you can use the `add_filter()` method of a lexer:
-
-.. sourcecode:: pycon
-
-    >>> from pygments.lexers import PythonLexer
-    >>> l = PythonLexer()
-    >>> # add a filter given by a string and options
-    >>> l.add_filter('codetagify', case='lower')
-    >>> l.filters
-    [<pygments.filters.CodeTagFilter object at 0xb785decc>]
-    >>> from pygments.filters import KeywordCaseFilter
-    >>> # or give an instance
-    >>> l.add_filter(KeywordCaseFilter(case='lower'))
-
-The `add_filter()` method takes keyword arguments which are forwarded to
-the constructor of the filter.
-
-To get a list of all registered filters by name, you can use the
-`get_all_filters()` function from the `pygments.filters` module that returns an
-iterable for all known filters.
-
-If you want to write your own filter, have a look at :doc:`Write your own filter
-<filterdevelopment>`.
-
-
-Builtin Filters
-===============
-
-.. pygmentsdoc:: filters