rugments 1.0.0.beta1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 26f2cc3f22f83d8cf8402e83519c5387099c994c
+  data.tar.gz: 02981428ba1aea39859eb53ff59328d445a08eaf
+SHA512:
+  metadata.gz: a141db121829a1bf723722cf20b2a1cfb2bb72615d4c06b999ade59a855d2b9f1ccedeb0327c3518137e8076713f9cd02261004321eb20b8fffbe80f185757d1
+  data.tar.gz: 8f776f6add6c85bad26d41b5218cea4bcdd0f03488f949185d4e31aac1dc70e74eab28a1a804c85309d2987420c6a878812a4db803174258c5c9dda708651bf6

data/LICENSE

@@ -0,0 +1,52 @@
+The MIT License (MIT)
+
+Copyright (c) 2012-2014 Jeanine Adkisson
+Copyright (c) 2014 Stefan Tatschner
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+
+Many of the lexers in this project are adaptations of those in Pygments
+(pygments.org). The license for Pygments is as follows:
+
+Copyright (c) 2006-2014 by the respective authors (see AUTHORS file).
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+* Redistributions of source code must retain the above copyright
+  notice, this list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright
+  notice, this list of conditions and the following disclaimer in the
+  documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,195 @@
+# Rugments
+
+[![Gem Version](https://badge.fury.io/rb/rugments.svg)](http://badge.fury.io/rb/rugments)
+
+Rouge is a pure-ruby syntax highlighter.  It can highlight over 60 languages, and output HTML or ANSI 256-color text.  Its HTML output is compatible with stylesheets designed for [pygments][].
+
+If you'd like to help out with this project, assign yourself something from the [issues][] page, and send me a pull request (even if it's not done yet!).  Bonus points for feature branches.  In particular, I would appreciate help with the following lexers, from someone who has more experience with the language than I do:
+
+* Delphi/Pascal
+
+Also, if anyone with design skills feels like helping me make a website for rouge, I'd really appreciate the help.  So far all I've got is the [demo page][pretty colors].
+
+[issues]: https://github.com/jneen/rouge/issues "Help Out"
+[pygments]: http://pygments.org/ "Pygments"
+
+## Usage
+
+First, take a look at the [pretty colors][].
+
+[pretty colors]: http://rouge.jayferd.us/demo
+
+``` ruby
+# make some nice lexed html
+source = File.read('/etc/bashrc')
+formatter = Rouge::Formatters::HTML.new(css_class: 'highlight')
+lexer = Rouge::Lexers::Shell.new
+formatter.format(lexer.lex(source))
+
+# Get some CSS
+Rouge::Themes::Base16.mode(:light).render(scope: '.highlight')
+# Or use Theme#find with string input
+Rouge::Theme.find('base16.light').render(scope: '.highlight')
+```
+
+### Full options
+#### Formatter options
+##### css_class: 'highlight'
+Apply a class to the syntax-highlighted output. Set to false to not apply any css class.
+
+##### line_numbers: false
+Generate line numbers.
+
+##### start_line: 1
+Index to start line numbers.
+
+##### inline_theme: nil
+A `Rouge::CSSTheme` used to highlight the output with inline styles instead of classes. Allows string inputs (separate mode with a dot):
+
+```
+%w[colorful github monokai monokai.sublime thankful_eyes base16
+   base16.dark base16.light base16.solarized base16.monokai]
+```
+
+##### wrap: true  
+Wrap the highlighted content in a container. Defaults to `<pre><code>`, or `<div>` if line numbers are enabled.
+
+#### Lexer options
+##### debug: false
+Print a trace of the lex on stdout
+
+##### parent: ''
+Allows you to specify which language the template is inside
+
+#### CSS theme options
+##### scope: '.highlight'  
+CSS selector that styles are applied to, e.g. `Rouge::Themes::Monokai.mode(:sublime).render(scope: 'code')`
+
+Rouge aims to be simple to extend, and to be a drop-in replacement for pygments, with the same quality of output. Also, Rouge ships with a `rougify` command which allows you to easily highlight files in your terminal:
+
+``` bash
+$ rougify foo.rb
+$ rougify style monokai.sublime > syntax.css
+```
+
+### Advantages to pygments.rb
+* No need to [spawn Python processes](https://github.com/tmm1/pygments.rb).
+
+### Advantages to CodeRay
+* The HTML output from Rouge is fully compatible with stylesheets designed for pygments.
+* The lexers are implemented with a dedicated DSL, rather than being hand-coded.
+* Rouge supports every language CodeRay does except for Pascal/Delphi (pull requests happily accepted!), and more.
+
+## You can even use it with Redcarpet
+
+``` ruby
+require 'redcarpet'
+require 'rouge'
+require 'rouge/plugins/redcarpet'
+
+class HTML < Redcarpet::Render::HTML
+  include Rouge::Plugins::Redcarpet # yep, that's it.
+end
+```
+
+If you have `:fenced_code_blocks` enabled, you can specify languages, and even options with CGI syntax, like `php?start_inline=1`, or `erb?parent=javascript`.
+
+## Encodings
+
+Rouge is only for UTF-8 strings.  If you'd like to highlight a string with a different encoding, please convert it to UTF-8 first.
+
+## Other integrations
+
+* Middleman: [middleman-syntax](https://github.com/middleman/middleman-syntax) (@bhollis)
+* Middleman: [middleman-rouge][] (@Linuus)
+* RDoc: [rdoc-rouge][] (@zzak)
+
+[middleman-rouge]: https://github.com/Linuus/middleman-rouge
+[rdoc-rouge]: https://github.com/zzak/rdoc-rouge
+
+## Contributing
+
+### Installing Ruby
+
+If you're here to implement a lexer for your awesome language, there's a good chance you don't already have a ruby development environment set up.  Follow the [instructions on the wiki](https://github.com/jneen/rouge/wiki/Setting-up-Ruby) to get up and running.  If you have trouble getting set up, let me know - I'm always happy to help.
+
+### Run the tests
+
+You can test the core of Rouge simply by running `rake` (no `bundle exec` required).  It's also set up with `guard`, if you like.
+
+To test a lexer visually, run `rackup` from the root and go to `localhost:9292/#{some_lexer}` where `some_lexer` is the tag or an alias of a lexer you'd like to test.  If you add `?debug=1`, helpful debugging info will be printed on stdout.
+
+### API Documentation
+
+is at http://rubydoc.info/gems/rouge/frames.
+
+### Using the lexer DSL
+
+You can probably learn a lot just by reading through the existing lexers.  Basically, a lexer consists of a collection of states, each of which has several rules.  A rule consists of a regular expression and an action, which yields tokens and manipulates the state stack.  Each rule in the state on top of the stack is tried *in order* until a match is found, at which point the action is run, the match consumed from the stream, and the process repeated with the new lexer on the top of the stack.  Each lexer has a special state called `:root`, and the initial state stack consists of just this state.
+
+Here's how you might use it:
+
+``` ruby
+class MyLexer < Rouge::RegexLexer
+  state :root do
+    # the "easy way"
+
+    # simple rules
+    rule /0x[0-9a-f]+/, Num::Hex
+
+    # simple state stack manipulation
+    rule /{-/, Comment, :next_state
+    rule /-}/, Comment, :pop!
+
+    # the "flexible way"
+    rule /abc/ do |m|
+      # m is the match, for accessing match groups manually
+
+      # you can do the following things:
+      pop!
+      push :another_state
+      push # assumed to be the current state
+      state? :some_state # check if the current state is :some_state
+      in_state? :some_state # check if :some_state is in the state stack
+
+      # yield a token.  if no second argument is supplied, the value is
+      # taken to be the whole match.
+      # The sum of all the tokens yielded must be equivalent to the whole
+      # match - otherwise characters will go missing from the user's input.
+      token Generic::Output, m[0]
+
+      # calls SomeOtherLexer.lex(str) and yields its output.  See the
+      # HTML lexer for a nice example of this.
+      # if no second argument is supplied, it is assumed to be the whole
+      # match string.
+      delegate SomeOtherLexer, str
+
+      # the context object is the lexer itself, so you can stash state here
+      @count ||= 0
+      @count += 1
+
+      # advanced: push a dynamically created anonymous state
+      push do
+        rule /.../, Generic::Output
+      end
+    end
+
+    rule /(\w+)(:)/
+      # "groups" yields the matched groups in order
+      groups Name::Label, Punctuation
+    end
+  end
+
+  start do
+    # this is run whenever a fresh lex is started
+  end
+end
+```
+
+## Tips
+
+I don't get paid to maintain rouge. If you've found this software useful, consider dropping a tip in the [bucket](http://www.gittip.com/jneen).
+
+## License
+
+Rouge is released under the MIT license. Please see the `LICENSE` file for more information.

data/bin/rugmentize

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+
+require "thor"
+
+class Cli < Thor
+end

data/lib/rugments/cli.rb

@@ -0,0 +1,357 @@
+# not required by the main lib.
+# to use this module, require 'rouge/cli'.
+
+module Rugments
+  class FileReader
+    attr_reader :input
+    def initialize(input)
+      @input = input
+    end
+
+    def file
+      case input
+      when '-'
+        $stdin
+      when String
+        File.new(input)
+      when ->(i){ i.respond_to? :read }
+        input
+      end
+    end
+
+    def read
+      @read ||= begin
+        file.read
+      rescue => e
+        $stderr.puts "unable to open #{input}: #{e.message}"
+        exit 1
+      ensure
+        file.close
+      end
+    end
+  end
+
+  class CLI
+    def self.doc
+      return enum_for(:doc) unless block_given?
+
+      yield %|usage: rougify [command] [args...]|
+      yield %||
+      yield %|where <command> is one of:|
+      yield %|	highlight	#{Highlight.desc}|
+      yield %|	help		#{Help.desc}|
+      yield %|	style		#{Style.desc}|
+      yield %|	list		#{List.desc}|
+      yield %|	version		#{Version.desc}|
+      yield %||
+      yield %|See `rougify help <command>` for more info.|
+    end
+
+    class Error < StandardError
+      attr_reader :message, :status
+      def initialize(message, status=1)
+        @message = message
+        @status = status
+      end
+    end
+
+    def self.parse(argv=ARGV)
+      argv = normalize_syntax(argv)
+
+      mode = argv.shift
+
+      klass = class_from_arg(mode)
+      return klass.parse(argv) if klass
+
+      case mode
+      when '-h', '--help', 'help', '-help'
+        Help.parse(argv)
+      else
+        argv.unshift(mode) if mode
+        Highlight.parse(argv)
+      end
+    end
+
+    def initialize(options={})
+    end
+
+    def self.error!(msg, status=1)
+      raise Error.new(msg, status)
+    end
+
+    def error!(*a)
+      self.class.error!(*a)
+    end
+
+    def self.class_from_arg(arg)
+      case arg
+      when 'version', '--version'
+        Version
+      when 'help'
+        Help
+      when 'highlight', 'hi'
+        Highlight
+      when 'style'
+        Style
+      when 'list'
+        List
+      end
+    end
+
+    class Version < CLI
+      def self.desc
+        "print the rouge version number"
+      end
+
+      def self.parse(*); new; end
+
+      def run
+        puts Rouge.version
+      end
+    end
+
+    class Help < CLI
+      def self.desc
+        "print help info"
+      end
+
+      def self.doc
+        return enum_for(:doc) unless block_given?
+
+        yield %|usage: rougify help <command>|
+        yield %||
+        yield %|print help info for <command>.|
+      end
+
+      def self.parse(argv)
+        opts = { :mode => CLI }
+        until argv.empty?
+          arg = argv.shift
+          klass = class_from_arg(arg)
+          if klass
+            opts[:mode] = klass
+            next
+          end
+        end
+        new(opts)
+      end
+
+      def initialize(opts={})
+        @mode = opts[:mode]
+      end
+
+      def run
+        @mode.doc.each(&method(:puts))
+      end
+    end
+
+    class Highlight < CLI
+      def self.desc
+        "highlight code"
+      end
+
+      def self.doc
+        return enum_for(:doc) unless block_given?
+
+        yield %[usage: rougify highlight <filename> [options...]]
+        yield %[       rougify highlight [options...]]
+        yield %[]
+        yield %[--input-file|-i <filename>  specify a file to read, or - to use stdin]
+        yield %[]
+        yield %[--lexer|-l <lexer>          specify the lexer to use.]
+        yield %[                            If not provided, rougify will try to guess]
+        yield %[                            based on --mimetype, the filename, and the]
+        yield %[                            file contents.]
+        yield %[]
+        yield %[--mimetype|-m <mimetype>    specify a mimetype for lexer guessing]
+        yield %[]
+        yield %[--lexer-opts|-L <opts>      specify lexer options in CGI format]
+        yield %[                            (opt1=val1&opt2=val2)]
+        yield %[]
+        yield %[--formatter-opts|-F <opts>  specify formatter options in CGI format]
+        yield %[                            (opt1=val1&opt2=val2)]
+      end
+
+      def self.parse(argv)
+        opts = {
+          :formatter => 'terminal256',
+          :input_file => '-',
+          :lexer_opts => {},
+          :formatter_opts => {},
+        }
+
+        until argv.empty?
+          arg = argv.shift
+          case arg
+          when '--input-file', '-i'
+            opts[:input_file] = argv.shift
+          when '--mimetype', '-m'
+            opts[:mimetype] = argv.shift
+          when '--lexer', '-l'
+            opts[:lexer] = argv.shift
+          when '--formatter', '-f'
+            opts[:formatter] = argv.shift
+          when '--lexer-opts', '-L'
+            opts[:lexer_opts] = parse_cgi(argv.shift)
+          when '--formatter-opts', '-F'
+            opts[:formatter_opts] = parse_cgi(argv.shift)
+          when /^--/
+            error! "unknown option #{arg.inspect}"
+          else
+            opts[:input_file] = arg
+          end
+        end
+
+        new(opts)
+      end
+
+      def input_stream
+        @input_stream ||= FileReader.new(@input_file)
+      end
+
+      def input
+        @input ||= input_stream.read
+      end
+
+      def lexer_class
+        @lexer_class ||= Lexer.guess(
+          :filename => @input_file,
+          :mimetype => @mimetype,
+          :source => input_stream,
+        )
+      end
+
+      def lexer
+        @lexer ||= lexer_class.new(@lexer_opts)
+      end
+
+      attr_reader :input_file, :lexer_name, :mimetype, :formatter
+
+      def initialize(opts={})
+        @input_file = opts[:input_file]
+
+        if opts[:lexer]
+          @lexer_class = Lexer.find(opts[:lexer]) \
+            or error! "unkown lexer #{opts[:lexer].inspect}"
+        else
+          @lexer_name = opts[:lexer]
+          @mimetype = opts[:mimetype]
+        end
+
+        @lexer_opts = opts[:lexer_opts]
+
+        formatter_class = Formatter.find(opts[:formatter]) \
+          or error! "unknown formatter #{opts[:formatter]}"
+
+        @formatter = formatter_class.new(opts[:formatter_opts])
+      end
+
+      def run
+        formatter.format(lexer.lex(input), &method(:print))
+      end
+
+    private
+      def self.parse_cgi(str)
+        pairs = CGI.parse(str).map { |k, v| [k.to_sym, v.first] }
+        Hash[pairs]
+      end
+    end
+
+    class Style < CLI
+      def self.desc
+        "print CSS styles"
+      end
+
+      def self.doc
+        return enum_for(:doc) unless block_given?
+
+        yield %|usage: rougify style [<theme-name>] [<options>]|
+        yield %||
+        yield %|Print CSS styles for the given theme.  Extra options are|
+        yield %|passed to the theme.  Theme defaults to thankful_eyes.|
+        yield %||
+        yield %|options:|
+        yield %|  --scope	(default: .highlight) a css selector to scope by|
+        yield %||
+        yield %|available themes:|
+        yield %|  #{Theme.registry.keys.sort.join(', ')}|
+      end
+
+      def self.parse(argv)
+        opts = { :theme_name => 'thankful_eyes' }
+
+        until argv.empty?
+          arg = argv.shift
+          case arg
+          when /--(\w+)/
+            opts[$1.tr('-', '_').to_sym] = argv.shift
+          else
+            opts[:theme_name] = arg
+          end
+        end
+
+        new(opts)
+      end
+
+      def initialize(opts)
+        theme_name = opts.delete(:theme_name)
+        theme_class = Theme.find(theme_name) \
+          or error! "unknown theme: #{theme_name}"
+
+        @theme = theme_class.new(opts)
+      end
+
+      def run
+        @theme.render(&method(:puts))
+      end
+    end
+
+    class List < CLI
+      def self.desc
+        "list available lexers"
+      end
+
+      def self.doc
+        return enum_for(:doc) unless block_given?
+
+        yield %|usage: rouge list|
+        yield %||
+        yield %|print a list of all available lexers with their descriptions.|
+      end
+
+      def self.parse(argv)
+        new
+      end
+
+      def run
+        puts "== Available Lexers =="
+
+        Lexer.all.sort_by(&:tag).each do |lexer|
+          desc = "#{lexer.desc}"
+          if lexer.aliases.any?
+            desc << " [aliases: #{lexer.aliases.join(',')}]"
+          end
+          puts "%s: %s" % [lexer.tag, desc]
+          puts
+        end
+      end
+    end
+
+  private
+    def self.normalize_syntax(argv)
+      out = []
+      argv.each do |arg|
+        case arg
+        when /^(--\w+)=(.*)$/
+          out << $1 << $2
+        when /^(-\w)(.+)$/
+          out << $1 << $2
+        else
+          out << arg
+        end
+      end
+
+      out
+    end
+  end
+end

data/lib/rugments/formatter.rb

@@ -0,0 +1,29 @@
+module Rugments
+  # A Formatter takes a token stream and formats it for human viewing.
+  class Formatter
+    REGISTRY = {}
+
+    # Specify or get the unique tag for this formatter.  This is used
+    # for specifying a formatter in `rougify`.
+    def self.tag(tag = nil)
+      return @tag unless tag
+      REGISTRY[tag] = self
+
+      @tag = tag
+    end
+
+    # Find a formatter class given a unique tag.
+    def self.find(tag)
+      REGISTRY[tag]
+    end
+
+    # Format a token stream.  Delegates to {#format}.
+    def self.format(tokens, opts = {}, &b)
+      new(opts).format(tokens, &b)
+    end
+  end
+end
+
+require_relative 'formatters/html'
+require_relative 'formatters/terminal256'
+require_relative 'formatters/null'