pygments.rb-jruby 0.5.4

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 497802c2d36d83721eff4650c2c487c68a2409c6
+  data.tar.gz: be724cc3acd969987cd64fe20c69e782fdad7424
+SHA512:
+  metadata.gz: f0ae7a8a302689694612f04bb3dc2f5179fe9c556649779048f2fa4e43dffc1ea11e6fcbdd324bdbafbbe2eaf76b53cd4f8047221af3ec7b5b2b347710fd40fe
+  data.tar.gz: 95bd0b88d624742f25aa8b5e411c2864fbedc5089cf8ef0f617bb2fa4ffc7ccda249a996f7382a2500b4c0f1eb0dd5bbe64ae1c709059b7b13176b4f87fde879

data/.gitignore

@@ -0,0 +1,6 @@
+Gemfile.lock
+ext/Makefile
+lib/pygments_ext.*
+tmp
+pkg
+*.pyc

data/CHANGELOG.md

@@ -0,0 +1,71 @@
+changelog
+===========
+
+Version 0.5.4 (Nov 3, 2013)
+-----------------------------
+
+* Update lexers file
+
+Version 0.5.3 (Sep 17, 2013)
+-----------------------------
+
+* Fixes for Slash lexer
+* Improve highlighting for Slash lexer
+* Upgrade to latest pygments (1.7, changes summary follows.  See pygments changelog for details)
+  * Add Clay lexer
+  * Add Perl 6 lexer
+  * Add Swig lexer
+  * Add nesC lexer
+  * Add BlitzBasic lexer
+  * Add EBNF lexer
+  * Add Igor Pro lexer
+  * Add Rexx lexer
+  * Add Agda lexer
+  * Recognize vim modelines
+  * Improve Python 3 lexer
+  * Improve Opa lexer
+  * Improve Julia lexer
+  * Improve Lasso lexer
+  * Improve Objective C/C++ lexer
+  * Improve Ruby lexer
+  * Improve Stan lexer
+  * Improve JavaScript lexer
+  * Improve HTTP lexer
+  * Improve Koka lexer
+  * Improve Haxe lexer
+  * Improve Prolog lexer
+  * Improve F# lexer
+
+Version 0.5.2 (July 17, 2013)
+-----------------------------
+
+* Add Slash lexer
+
+Version 0.5.1 (June 25, 2013)
+-----------------------------
+
+* Ensure compatability across distros by detecting if `python2` is available
+
+Version 0.5.0 (Apr 13, 2013)
+-----------------------------
+
+* Use #rstrip to fix table mode bug
+
+Version 0.4.2 (Feb 25, 2013)
+-----------------------------
+
+* Add new lexers, including custom lexers
+
+Version 0.3.7 (Jan 2, 2013)
+-----------------------------
+
+* Fixed missing custom lexers
+* Added syntax highlighting for Hxml
+
+Version 0.3.4 (Dec 28, 2012)
+-----------------------------
+
+* Add support for Windows
+* Add MIT license
+
+

data/Gemfile

@@ -0,0 +1,2 @@
+source "https://rubygems.org"
+gemspec

data/LICENSE

@@ -0,0 +1,17 @@
+The MIT License (MIT)
+Copyright (c) Ted Nyman and Aman Gupta, 2012-2013
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and 
+associated documentation files (the "Software"), to deal in the Software without restriction, 
+including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, 
+and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, 
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial 
+portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT 
+LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,118 @@
+# pygments.rb
+
+A Ruby wrapper for the Python [pygments syntax highlighter](http://pygments.org/).
+
+pygments.rb works by talking over a simple pipe to a long-lived 
+Python child process. This library replaces [github/albino](https://github.com/github/albino),
+as well as a version of pygments.rb that used an embedded Python
+interpreter. 
+
+Each Ruby process that runs has its own 'personal Python'; 
+for example, 4 Unicorn workers will have one Python process each. 
+If a Python process dies, a new one will be spawned on the next 
+pygments.rb request.
+
+## system requirements
+
+- Python 2.5, Python 2.6, or Python 2.7. You can always use Python 2.x from a `virtualenv` if
+  your default Python install is 3.x.
+
+## usage
+
+``` ruby 
+require 'pygments'
+``` 
+
+``` ruby
+Pygments.highlight(File.read(__FILE__), :lexer => 'ruby')
+```
+
+Encoding and other lexer/formatter options can be passed in via an
+options hash:
+
+``` ruby
+Pygments.highlight('code', :options => {:encoding => 'utf-8'})
+```
+
+pygments.rb defaults to using an HTML formatter. 
+To use a formatter other than `html`, specify it explicitly
+like so:
+
+``` ruby
+Pygments.highlight('code', :formatter => 'bbcode')
+Pygments.highlight('code', :formatter => 'terminal')
+```
+
+To generate CSS for HTML formatted code, use the `#css` method:
+
+``` ruby
+Pygments.css
+Pygments.css('.highlight')
+```
+
+To use a specific pygments style, pass the `:style` option to the `#css` method:
+
+``` ruby
+Pygments.css(:style => "monokai")
+```
+
+Other Pygments high-level API methods are also available.
+These methods return arrays detailing all the available lexers, formatters, 
+and styles.
+
+``` ruby
+Pygments.lexers
+Pygments.formatters
+Pygments.styles
+```
+
+To use a custom pygments installation, specify the path to
+`Pygments#start`:
+
+``` ruby
+Pygments.start("/path/to/pygments")
+```
+
+If you'd like logging, set the environmental variable `MENTOS_LOG` to a file path for your logfile.
+
+## benchmarks
+
+
+    $ ruby bench.rb 50
+       Benchmarking....
+       Size: 698 bytes
+       Iterations: 50
+                                                      user     system      total        real
+       pygments popen                                0.010000   0.010000   0.020000 (  0.460370)
+       pygments popen (process already started)      0.010000   0.000000   0.010000 (  0.272975)
+       pygments popen (process already started 2)    0.000000   0.000000   0.000000 (  0.273589)
+
+    $ ruby bench.rb 10
+       Benchmarking....
+       Size: 15523 bytes
+       Iterations: 10
+                                                      user     system      total        real
+       pygments popen                               0.000000   0.000000   0.000000 (  0.819419)
+       pygments popen (process already started)     0.010000   0.000000   0.010000 (  0.676515)
+       pygments popen (process already started 2)   0.000000   0.010000   0.010000 (  0.674189)
+
+## license
+
+The MIT License (MIT)
+
+Copyright (c) Ted Nyman and Aman Gupta, 2012-2013
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and 
+associated documentation files (the "Software"), to deal in the Software without restriction, 
+including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, 
+and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, 
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial 
+portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT 
+LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, 
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,66 @@
+#!/usr/bin/env rake
+require "bundler/gem_tasks"
+
+task :default => :test
+
+# ==========================================================
+# Packaging
+# ==========================================================
+
+GEMSPEC = eval(File.read('pygments.rb.gemspec'))
+
+require 'rubygems/package_task' 
+
+# ==========================================================
+# Testing
+# ==========================================================
+
+require 'rake/testtask'
+Rake::TestTask.new 'test' do |t|
+  t.test_files = FileList['test/test_*.rb']
+  t.ruby_opts = ['-rubygems']
+end
+
+# ==========================================================
+# Benchmarking
+# ==========================================================
+
+task :bench do
+  sh "ruby bench.rb"
+end
+
+# ==========================================================
+# Cache lexers
+# ==========================================================
+
+# Write all the lexers to a file for easy lookup
+task :lexers do
+  sh "ruby cache-lexers.rb"
+end
+
+# ==========================================================
+# Vendor
+# ==========================================================
+
+namespace :vendor do
+  file 'vendor/pygments-main' do |f|
+    sh "hg clone https://bitbucket.org/birkenfeld/pygments-main #{f.name}"
+    sh "hg --repository #{f.name} identify --id > #{f.name}/REVISION"
+    rm_rf Dir["#{f.name}/.hg*"]
+  end
+
+  task :clobber do
+    rm_rf 'vendor/pygments-main'
+  end
+
+  # Load all the custom lexers in the `vendor/custom_lexers` folder
+  # and stick them in our custom Pygments vendor
+  task :load_lexers do
+    LEXERS_DIR = 'vendor/pygments-main/pygments/lexers'
+    lexers = FileList['vendor/custom_lexers/*.py']
+    lexers.each { |l| FileUtils.copy l, LEXERS_DIR }
+    FileUtils.cd(LEXERS_DIR) { sh "python _mapping.py" }
+  end
+
+  task :update => [:clobber, 'vendor/pygments-main', :load_lexers]
+end

data/bench.rb

@@ -0,0 +1,22 @@
+require File.join(File.dirname(__FILE__), '/lib/pygments.rb')
+require 'benchmark'
+
+include Benchmark
+# number of iterations
+num = ARGV[0] ? ARGV[0].to_i : 10
+
+# we can also repeat the code itself
+repeats = ARGV[1] ? ARGV[1].to_i : 1
+
+code = File.open('test/test_data.py').read.to_s * repeats
+
+puts "Benchmarking....\n"
+puts "Size: " + code.bytesize.to_s + " bytes\n"
+puts "Iterations: " + num.to_s + "\n"
+
+Benchmark.bm(40) do |x|
+  x.report("pygments popen                             ")  { for i in 1..num; Pygments.highlight(code, :lexer => 'python'); end }
+  x.report("pygments popen (process already started)   ")  { for i in 1..num; Pygments.highlight(code, :lexer => 'python'); end }
+  x.report("pygments popen (process already started 2) ")  { for i in 1..num; Pygments.highlight(code, :lexer => 'python'); end }
+end
+

data/cache-lexers.rb

@@ -0,0 +1,8 @@
+require File.join(File.dirname(__FILE__), '/lib/pygments.rb')
+
+# Simple marshalling
+serialized_lexers = Marshal.dump(Pygments.lexers!)
+
+# Write to a file
+File.open("lexers", 'w') { |file| file.write(serialized_lexers) }
+

data/lib/pygments.rb

@@ -0,0 +1,8 @@
+require File.join(File.dirname(__FILE__), 'pygments/popen')
+
+
+module Pygments
+  extend Pygments::Popen
+
+  autoload :Lexer, 'pygments/lexer'
+end

data/lib/pygments/lexer.rb

@@ -0,0 +1,148 @@
+module Pygments
+  class Lexer < Struct.new(:name, :aliases, :filenames, :mimetypes)
+    @lexers          = []
+    @index           = {}
+    @name_index      = {}
+    @alias_index     = {}
+    @extname_index   = {}
+    @mimetypes_index = {}
+
+    # Internal: Create a new Lexer object
+    #
+    # hash - A hash of attributes
+    #
+    # Returns a Lexer object
+    def self.create(hash)
+      lexer = new(hash[:name], hash[:aliases], hash[:filenames], hash[:mimetypes])
+
+      @lexers << lexer
+
+      @index[lexer.name.downcase] = @name_index[lexer.name] = lexer
+
+      lexer.aliases.each do |name|
+        @alias_index[name] = lexer
+        @index[name.downcase] ||= lexer
+      end
+
+      lexer.filenames.each do |filename|
+        extnames = []
+
+        extname = File.extname(filename)
+        if m = extname.match(/\[(.+)\]/)
+          m[1].scan(/./).each do |s|
+            extnames << extname.sub(m[0], s)
+          end
+        elsif extname != ""
+          extnames << extname
+        end
+
+        extnames.each do |extname|
+          @extname_index[extname] = lexer
+          @index[extname.downcase.sub(/^\./, "")] ||= lexer
+        end
+      end
+
+      lexer.mimetypes.each do |type|
+        @mimetypes_index[type] = lexer
+      end
+
+      lexer
+    end
+
+    # Public: Get all Lexers
+    #
+    # Returns an Array of Lexers
+    def self.all
+      @lexers
+    end
+
+    # Public: Look up Lexer by name or alias.
+    #
+    # name - A String name or alias
+    #
+    #   Lexer.find('Ruby')
+    #   => #<Lexer name="Ruby">
+    #
+    # Returns the Lexer or nil if none was found.
+    def self.find(name)
+      @index[name.to_s.downcase]
+    end
+
+    # Public: Alias for find.
+    def self.[](name)
+      find(name)
+    end
+
+    # Public: Look up Lexer by its proper name.
+    #
+    # name - The String name of the Lexer
+    #
+    # Examples
+    #
+    #   Lexer.find_by_name('Ruby')
+    #   # => #<Lexer name="Ruby">
+    #
+    # Returns the Lexer or nil if none was found.
+    def self.find_by_name(name)
+      @name_index[name]
+    end
+
+    # Public: Look up Lexer by one of its aliases.
+    #
+    # name - A String alias of the Lexer
+    #
+    # Examples
+    #
+    #   Lexer.find_by_alias('rb')
+    #   # => #<Lexer name="Ruby">
+    #
+    # Returns the Lexer or nil if none was found.
+    def self.find_by_alias(name)
+      @alias_index[name]
+    end
+
+    # Public: Look up Lexer by one of it's file extensions.
+    #
+    # extname - A String file extension.
+    #
+    # Examples
+    #
+    #  Lexer.find_by_extname('.rb')
+    #  # => #<Lexer name="Ruby">
+    #
+    # Returns the Lexer or nil if none was found.
+    def self.find_by_extname(extname)
+      @extname_index[extname]
+    end
+
+    # Public: Look up Lexer by one of it's mime types.
+    #
+    # type - A mime type String.
+    #
+    # Examples
+    #
+    #  Lexer.find_by_mimetype('application/x-ruby')
+    #  # => #<Lexer name="Ruby">
+    #
+    # Returns the Lexer or nil if none was found.
+    def self.find_by_mimetype(type)
+      @mimetypes_index[type]
+    end
+
+    # Public: Highlight syntax of text
+    #
+    # text    - String of code to be highlighted
+    # options - Hash of options (defaults to {})
+    #
+    # Returns html String
+    def highlight(text, options = {})
+      options[:lexer] = aliases.first
+      Pygments.highlight(text, options)
+    end
+
+    alias_method :==, :equal?
+    alias_method :eql?, :equal?
+  end
+
+  lexers.values.each { |h| Lexer.create(h) }
+end