mustermann 0.0.1 → 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 9a623efd27ea171ba18818550616788aeca18ded
+  data.tar.gz: f8e7956944902e37495c44085662cb39cedea8a9
+SHA512:
+  metadata.gz: f8e52f5e109283a3f6d9d2823b1410e3e41e3d8c3d3787e767c70fa8e7a018ff53738f994d856947c2473f8ef94c65eb56de79f51132aa729ca26a4d582ada2f
+  data.tar.gz: f3ff266371da60a7da88f492ad2ee3a46389a1a4806c2e57886f07b7b702408388ffaf3ff18175d59c1b67fdfafb2eb5a07d496ce8219cdccc3ae5fd90f06203

data/.gitignore

@@ -3,10 +3,11 @@
 .bundle
 .config
 .yardoc
+.test_queue_stats
+.coverage
 Gemfile.lock
 InstalledFiles
 _yardoc
-coverage
 doc/
 lib/bundler/man
 pkg

data/.travis.yml

@@ -1,6 +1,7 @@
 rvm:
   - 2.0.0
   - ruby-head
-matrix:
-  allow_failures:
-    - rvm: ruby-head
+#  - jruby-head
+#matrix:
+#  allow_failures:
+#    - rvm: jruby-head

data/.yardopts

@@ -0,0 +1 @@
+- README.md internals.md

data/README.md

@@ -1,10 +1,8 @@
-# Mustermann
+# The Amazing Mustermann
 
-[![Build Status](https://travis-ci.org/rkh/mustermann.png?branch=master)](https://travis-ci.org/rkh/mustermann)
-[![Coverage Status](https://coveralls.io/repos/rkh/mustermann/badge.png?branch=master)](https://coveralls.io/r/rkh/mustermann)
-[![Code Climate](https://codeclimate.com/github/rkh/mustermann.png)](https://codeclimate.com/github/rkh/mustermann)
-[![Dependency Status](https://gemnasium.com/rkh/mustermann.png)](https://gemnasium.com/rkh/mustermann)
-[![Gem Version](https://badge.fury.io/rb/mustermann.png)](http://badge.fury.io/rb/mustermann)
+[![Build Status](https://travis-ci.org/rkh/mustermann.png?branch=master)](https://travis-ci.org/rkh/mustermann) [![Coverage Status](https://coveralls.io/repos/rkh/mustermann/badge.png?branch=master)](https://coveralls.io/r/rkh/mustermann) [![Code Climate](https://codeclimate.com/github/rkh/mustermann.png)](https://codeclimate.com/github/rkh/mustermann) [![Dependency Status](https://gemnasium.com/rkh/mustermann.png)](https://gemnasium.com/rkh/mustermann) [![Gem Version](https://badge.fury.io/rb/mustermann.png)](http://badge.fury.io/rb/mustermann)
+
+*Make sure you view the correct docs: [latest release](http://rubydoc.info/gems/mustermann/frames), [master](http://rubydoc.info/github/rkh/mustermann/master/frames).*
 
 Welcome to [Mustermann](http://en.wikipedia.org/wiki/List_of_placeholder_names_by_language#German). Mustermann is your personal string matching expert. As an expert in the field of strings and patterns, Mustermann also has no runtime dependencies and is fully covered with specs and documentation.
 
@@ -29,14 +27,22 @@ pattern = Mustermann.new('/:prefix/*.*')
 pattern.params('/a/b.c') # => { "prefix" => "a", splat => ["b", "c"] }
 ```
 
-It's generally a good idea to reuse pattern objects, since as much computation as possible is happening during object creation, so that the actual matching is quite fast.
+Similarly, it is also possible to generate a string from a pattern by expanding it with such a hash:
+
+``` ruby
+pattern = Mustermann.new('/:file(.:ext)?')
+pattern.expand(file: 'pony')             # => "/pony"
+pattern.expand(file: 'pony', ext: 'jpg') # => "/pony.jpg"
+```
+
+It's generally a good idea to reuse pattern objects, since as much computation as possible is happening during object creation, so that the actual matching or expanding is quite fast.
 
 ## Types and Options
 
 You can pass in additional options to take fine grained control over the pattern:
 
 ``` ruby
-Mustermann.new('/:foo.:bar', capture: :alpha) # :foo and :bar will only match alphabetic character
+Mustermann.new('/:foo.:bar', capture: :alpha) # :foo and :bar will only match alphabetic characters
 ```
 
 In fact, you can even completely change the pattern type:
@@ -639,6 +645,43 @@ you should set `uri_decode` to `false` in order to conform with the specificatio
 
 If you are looking for an alternative implementation that also supports expanding, check out [addressable](http://addressable.rubyforge.org/).
 
-## Versioning
+## Requirements
+
+Mustermann has no dependencies besides a Ruby 2.0 compatible Ruby implementation.
+
+It is known to work on **MRI 2.0** and **MRI trunk**.
+**JRuby** is not yet supported, but is likely to follow soon (see [issue #2](https://github.com/rkh/mustermann/issues/2) for up to date information on JRuby).
+**Rubinius** is not yet able to parse the Mustermann source code.
+
+## Release History
+
+Mustermann follows [Semantic Versioning 2.0](http://semver.org/). Anything documented in the README or via YARD and not declared private is part of the public API.
+
+### Stable Releases
+
+There have been no stable releases yet. The code base is considered solid but I don't know of anyone using it in production yet.
+As there has been no stable release yet, the API might still change, though I consider this unlikely.
+
+### Development Releases
+
+* **Mustermann 0.0.1** (2013-04-27)
+    * More Infos:
+      [RubyGems.org](http://rubygems.org/gems/mustermann/versions/0.0.1),
+      [RubyDoc.info](rubydoc.info/gems/mustermann/0.0.1/frames),
+      [GitHub.com](https://github.com/rkh/mustermann/tree/v0.0.1)
+    * Initial Release.
+* **Mustermann 0.1.0** (2013-05-12)
+    * Add `Pattern#expand` for generating strings from patterns.
+    * Add better internal API for working with the AST.
+    * Improved documentation.
+    * Avoids parsing the path twice when used as Sinatra extension.
+    * Better exceptions for unknown pattern types.
+    * Better handling of edge cases around extend.
+    * More specs to ensure API stability.
+    * Largely rework internals of Sinatra, Rails and Template patterns.
+
+### Upcoming Releases
 
-Mustermann follows [Semantic Versioning](http://semver.org/).
+* **Mustermann 0.2.0** (next release with new features)
+* **Mustermann 1.0.0** (before Sinatra 2.0)
+    * First stable release.

data/Rakefile

@@ -1,3 +1,6 @@
+ENV['JRUBY_OPTS'] = '--2.0'
+ENV['RBXOPT'] = '-X20'
+
 task(:spec) { ruby '-w -S rspec' }
 task(:doc_stats) { ruby '-S yard stats' }
-task(default: [:spec, :doc_stats])
+task(default: [:spec, :doc_stats])

data/bench/capturing.rb

@@ -2,23 +2,56 @@ $:.unshift File.expand_path('../lib', __dir__)
 
 require 'benchmark'
 require 'mustermann'
+require 'mustermann/regexp_based'
 require 'addressable/template'
 
+
+Mustermann.register(:regexp, Class.new(Mustermann::RegexpBased) {
+  def compile(string, **options)
+    Regexp.new(string)
+  end
+}, load: false)
+
+Mustermann.register(:addressable, Class.new(Mustermann::RegexpBased) {
+  def compile(string, **options)
+    Addressable::Template.new(string)
+  end
+}, load: false)
+
 list = [
-  /\A\/(?<splat>.*?)\/(?<name>[^\/\?#]+)\Z/,
-  Mustermann.new('/*/:name',          type: :sinatra),
-  Mustermann.new('/*/:name',          type: :simple),
-  Mustermann.new('/*prefix/:name',    type: :rails),
-  Mustermann.new('{/prefix*}/{name}', type: :template),
-  #Addressable::Template.new('{/prefix*}/{name}')
+  [:sinatra,     '/*/:name'                                ],
+  [:rails,       '/*prefix/:name'                          ],
+  [:simple,      '/*/:name'                                ],
+  [:template,    '{/prefix*}/{name}'                       ],
+  [:regexp,      '\A\/(?<splat>.*?)\/(?<name>[^\/\?#]+)\Z' ],
+  [:addressable, '{/prefix*}/{name}'                       ]
 ]
 
+def self.assert(value)
+  fail unless value
+end
+
 string = '/a/b/c/d'
+name   = 'd'
+
+GC.disable
+
+puts "Compilation:"
+Benchmark.bmbm do |x|
+  list.each do |type, pattern|
+    x.report(type) { 1_000.times { Mustermann.new(pattern, type: type) } }
+  end
+end
 
+puts "", "Matching with two captures (one splat, one normal):"
 Benchmark.bmbm do |x|
-  list.each do |pattern|
-    x.report pattern.class.to_s do
-      100_000.times { pattern.match(string).captures }
+  list.each do |type, pattern|
+    pattern = Mustermann.new(pattern, type: type)
+    x.report type do
+      10_000.times do
+        match = pattern.match(string)
+        assert match[:name] == name
+      end
     end
   end
 end

data/bench/template_vs_addressable.rb

@@ -18,6 +18,9 @@ require 'addressable/template'
     explode = klass.new("{/segments*}")
     x.report("explode, match") { 1_000.times { explode.match("/a/b/c").captures } }
     x.report("explode, miss") { 1_000.times { explode.match("/a/b/c.miss") } }
+
+    expand = klass.new("/prefix/{foo}/something/{bar}")
+    x.report("expand") { 100.times { expand.expand(foo: 'foo', bar: 'bar').to_s } }
   end
   puts
 end

data/internals.md

@@ -0,0 +1,64 @@
+# Internal API
+
+This document describes how to use [Mustermann](README.md)'s internal API.
+
+It is a secondary goal to keep the internal API as stable as possible, in a state where it would well be possible to interface with it.
+However, the internal API is not covered by Semantic Versioning. As a rule of thumb, no backwards incompatible changes should be introduced to the API in minor releases (starting from 1.0.0).
+
+Should the internal API gain widespread/production use, we might consider moving parts of it over into the public API.
+
+Here is a quick example of what you can do with this:
+
+``` ruby
+require 'mustermann/ast/pattern'
+
+class MyPattern < Mustermann::AST::Pattern
+  on("~") { |c| node(:capture,     buffer[1]) if expect(/\{(\w+)\}/) }
+  on("+") { |c| node(:named_splat, buffer[1]) if expect(/\{(\w+)\}/) }
+  on("?") { |c| node(:optional, node(:capture, buffer[1])) if expect(/\{(\w+)\}/) }
+end
+
+pattern = MyPattern.new("/+{prefix}/~{page}/?{optional}")
+pattern.params("/a/")     # => nil
+pattern.params("/a/b/")   # => { "prefix" => "a",   "page" => "b", "optional" => nil }
+pattern.params("/a/b/c")  # => { "prefix" => "a",   "page" => "b", "optional" => "c" }
+pattern.params("/a/b/c/") # => { "prefix" => "a/b", "page" => "c", "optional" => nil }
+
+pattern.expand(prefix: "a",   page: "foo") # => "/a/foo/"
+pattern.expand(prefix: "a/b", page: "c/d") # => "/a/b/c%2Fd/"
+
+require 'mustermann'
+Mustermann.register(:my_pattern, MyPattern, load: false)
+Mustermann.new('/+{prefix}/~{page}/?{optional}', type: :my_pattern) # => #<MyPattern:"/+{prefix}/~{page}/?{optional}">
+
+require 'sinatra/base'
+class MyApp < Sinatra::Base
+  register Mustermann
+  set :pattern, type: :my_pattern
+
+  get '/hello/~{name}' do
+    "Hello #{params[:name].capitalize}!"
+  end
+end
+
+require 'mustermann/ast/tree_renderer'
+ast = MyPattern::Parser.parse(pattern.to_s)
+puts Mustermann::AST::TreeRenderer.render(ast)
+
+```
+
+## Pattern Registration
+
+...
+
+## Build Your Own Pattern
+
+...
+
+## Patterns Based on Regular Expressions
+
+...
+
+## AST-Based Patterns
+
+...

data/lib/mustermann.rb

@@ -1,18 +1,27 @@
 # Namespace and main entry point for the Mustermann library.
 #
-# Under normal circumstences the only external API entry point you should be using is {Mustermann.new}.
+# Under normal circumstances the only external API entry point you should be using is {Mustermann.new}.
 module Mustermann
-  # @param [String] string The string represenation of the new pattern
+  # @param [String] string The string representation of the new pattern
   # @param [Hash] options The options hash
   # @return [Mustermann::Pattern] pattern corresponding to string.
+  # @raise (see [])
+  # @raise (see Mustermann::Pattern.new)
   # @see file:README.md#Types_and_Options "Types and Options" in the README
   def self.new(string, type: :sinatra, **options)
     options.any? ? self[type].new(string, **options) : self[type].new(string)
   end
 
-  # @!visibility private
+  # Maps a type to its factory.
+  #
+  # @example
+  #   Mustermann[:sinatra] # => Mustermann::Sinatra
+  #
+  # @param [Symbol] key a pattern type identifier
+  # @raise [ArgumentError] if the type is not supported
+  # @return [Class, #new] pattern factory
   def self.[](key)
-    constant, library = register.fetch(key)
+    constant, library = register.fetch(key) { raise ArgumentError, "unsupported type %p" % key }
     require library if library
     constant.respond_to?(:new) ? constant : register[key] = const_get(constant)
   end
@@ -26,7 +35,7 @@ module Mustermann
 
   # @!visibility private
   def self.extend_object(object)
-    return super unless defined? ::Sinatra::Base and object < ::Sinatra::Base
+    return super unless defined? ::Sinatra::Base and object.is_a? Class and object < ::Sinatra::Base
     require 'mustermann/extension'
     object.register Extension
   end

data/lib/mustermann/ast/compiler.rb

@@ -0,0 +1,150 @@
+require 'mustermann/ast/translator'
+
+module Mustermann
+  # @see Mustermann::AST::Pattern
+  module AST
+    # Regexp compilation logic.
+    # @!visibility private
+    class Compiler < Translator
+      raises CompileError
+
+      # Trivial compilations
+      translate(Array)      { |**o| map { |e| t(e, **o) }.join  }
+      translate(:node)      { |**o| t(payload, **o)             }
+      translate(:separator) { |**o| Regexp.escape(payload)      }
+      translate(:optional)  { |**o| "(?:%s)?" % t(payload, **o) }
+      translate(:char)      { |**o| t.encoded(payload, **o)     }
+
+      translate :expression do |greedy: true, **options|
+        t(payload, allow_reserved: operator.allow_reserved, greedy: greedy && !operator.allow_reserved,
+          parametric: operator.parametric, separator: operator.separator, **options)
+      end
+
+      translate :with_look_ahead do |**options|
+        lookahead = each_leaf.inject("") do |ahead, element|
+          ahead + t(element, skip_optional: true, lookahead: ahead, greedy: false, no_captures: true, **options).to_s
+        end
+        lookahead << (at_end ? '$' : '/')
+        t(head, lookahead: lookahead, **options) + t(payload, **options)
+      end
+
+      # Capture compilation is complex. :(
+      # @!visibility private
+      class Capture < NodeTranslator
+        register :capture
+
+        # @!visibility private
+        def translate(**options)
+          return pattern(options) if options[:no_captures]
+          "(?<#{name}>#{translate(no_captures: true, **options)})"
+        end
+
+        # @return [String] regexp without the named capture
+        # @!visibility private
+        def pattern(capture: nil, **options)
+          case capture
+          when Symbol then from_symbol(capture, **options)
+          when Array  then from_array(capture, **options)
+          when Hash   then from_hash(capture, **options)
+          when String then from_string(capture, **options)
+          when nil    then from_nil(**options)
+          else capture
+          end
+        end
+
+        private
+          def qualified(string, greedy: true, **options)         "#{string}+#{?? unless greedy}"                                     end
+          def with_lookahead(string, lookahead: nil, **options)  lookahead ? "(?:(?!#{lookahead})#{string})" : string                end
+          def from_hash(hash,     **options)                     pattern(capture: hash[name.to_sym], **options)                      end
+          def from_array(array,   **options)                     Regexp.union(*array.map { |e| pattern(capture: e, **options) })     end
+          def from_symbol(symbol, **options)                     qualified(with_lookahead("[[:#{symbol}:]]", **options), **options)  end
+          def from_string(string, **options)                     Regexp.new(string.chars.map { |c| t.encoded(c, **options) }.join)   end
+          def from_nil(**options)                                qualified(with_lookahead(default(**options), **options), **options) end
+          def default(**options) "[^/\\?#]" end
+      end
+
+      # @!visibility private
+      class Splat < Capture
+        register :splat, :named_splat
+        # splats are always non-greedy
+        # @!visibility private
+        def pattern(**options)
+          ".*?"
+        end
+      end
+
+      # @!visibility private
+      class Variable < Capture
+        register :variable
+
+        # @!visibility private
+        def translate(**options)
+          return super(**options) if explode or not options[:parametric]
+          parametric super(parametric: false, **options)
+        end
+
+        # @!visibility private
+        def pattern(parametric: false, separator: nil, **options)
+          register_param(parametric: parametric, separator: separator, **options)
+          pattern = super(**options)
+          pattern = parametric(pattern) if parametric
+          pattern = "#{pattern}(?:#{Regexp.escape(separator)}#{pattern})*" if explode and separator
+          pattern
+        end
+
+        # @!visibility private
+        def parametric(string)
+          "#{Regexp.escape(name)}(?:=#{string})?"
+        end
+
+        # @!visibility private
+        def qualified(string, **options)
+          prefix ? "#{string}{1,#{prefix}}" : super(string, **options)
+        end
+
+        # @!visibility private
+        def default(allow_reserved: false, **options)
+          allow_reserved ? '[\w\-\.~%\:/\?#\[\]@\!\$\&\'\(\)\*\+,;=]' : '[\w\-\.~%]'
+        end
+
+        # @!visibility private
+        def register_param(parametric: false, split_params: nil, separator: nil, **options)
+          return unless explode and split_params
+          split_params[name] = { separator: separator, parametric: parametric }
+        end
+      end
+
+      # @return [String] Regular expression for matching the given character in all representations
+      # @!visibility private
+      def encoded(char, uri_decode: true, space_matches_plus: true, **options)
+        return Regexp.escape(char) unless uri_decode
+        encoded = escape(char, escape: /./)
+        list    = [escape(char), encoded.downcase, encoded.upcase].uniq.map { |c| Regexp.escape(c) }
+        list << encoded('+') if space_matches_plus and char == " "
+        "(?:%s)" % list.join("|")
+      end
+
+      # Compiles an AST to a regular expression.
+      # @param [Mustermann::AST::Node] ast the tree
+      # @return [Regexp] corresponding regular expression.
+      #
+      # @!visibility private
+      def self.compile(ast, **options)
+        new.compile(ast, **options)
+      end
+
+      # Compiles an AST to a regular expression.
+      # @param [Mustermann::AST::Node] ast the tree
+      # @return [Regexp] corresponding regular expression.
+      #
+      # @!visibility private
+      def compile(ast, except: nil, **options)
+        except   &&= "(?!#{translate(except, no_captures: true, **options)}\\Z)"
+        expression = "\\A#{except}#{translate(ast, **options)}\\Z"
+        Regexp.new(expression)
+      end
+    end
+
+    private_constant :Compiler
+  end
+end