cannonbol 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: f21caaed4e70356c5f043a524241bf103d8eddd8
+  data.tar.gz: 4ce959eb2fa8b569860d59bbb3de119b2b5c11ae
+SHA512:
+  metadata.gz: 98cf5aa3d8ea84c36503b5cfde8d22f833957ac44c650bf0629bf8ca44b672cc56d66408e0ce732ebc83c6ee3d71923a7497ca7c2cecd70fa8e37e149db0f5b4
+  data.tar.gz: 0ac7d0acbb284131ab62eca5456ad3152877949c51282cf40cc11a896fb1e752028bc1f66f036e0e4157e35a4afca933c936c3846212c2d3cb175c7bf4b494a8

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,3 @@
+language: ruby
+rvm:
+  - 2.1.1

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,13 @@
+# Contributor Code of Conduct
+
+As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, age, or religion.
+
+Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.
+
+This Code of Conduct is adapted from the [Contributor Covenant](http:contributor-covenant.org), version 1.0.0, available at [http://contributor-covenant.org/version/1/0/0/](http://contributor-covenant.org/version/1/0/0/)

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in cannonbol.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2015 catmando
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,420 @@
+# CannonBol
+
+CannonBol is a ruby DSL for patten matching based on SNOBOL4 and SPITBOL.
+
+* Makes complex patterns easier to read and write!
+* Combine regexes, plain strings and powerful new primitive match functions!
+* Makes capturing match results easy!
+* Allows recursive patterns!
+* Complete SNOBOL4 + SPITBOL extensions!
+* Based on the well documented, proven SNOBOL4 language!
+* Simple syntax looks great alongside ruby!
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'cannonbol'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install cannonbol
+
+## Lets Go!
+
+### Basic Matching `- &, |, capture?, match_any, match_all`
+
+Strings, Regexes and primitives are combined using & (concatenation) and | (alternation) operators
+
+Here is a simple pattern that matches a simple noun clause:
+
+    ("a" | "the") & /\s+/ & ("boy" | "girl")
+    
+So we will match either "a" or "the" followed white space and then by "boy or "girl".  Okay!  Lets use it!
+
+    ("a" | "the") & /\s+/ & ("boy" | "girl").match?("he saw a boy going home")
+    => "a boy"
+    ("a" | "the") & /\s+/ & ("boy" | "girl").match?("he saw a big boy going home")
+    => nil
+
+Now let's save the pieces of the match using the capture? (pronounced _capture IF_) method:
+
+    article, noun = nil, nil
+    pattern = ("a" | "the").capture? { |m| article = m } & /\s+/ & ("boy" | "girl").capture? { |m| noun = m }
+    pattern.match?("he saw the girl going home")
+    noun
+    => girl
+    article
+    => the
+
+The capture? method and its friend capture! (pronounced _capture NOW_) have many powerful features. As shown above it can take a block which is passed the matching substring, _IF the match succeeds_.  The other features of the capture method will be detailed [below.](Advanced capture techniques)
+
+Arrays can be turned into patterns using the match_any and match_all methods:
+
+    ARTICLES = ["a", "the"]
+    NOUNS = ["boy", "girl", "dog", "cat"]
+    ADJECTIVES = ["big", "small", "fierce", "friendly"]
+    WS = /\s+/
+    [ARTICLES.match_any, [WS, [WS, ADJECTIVES.match_any, WS].match_all].match_any, NOUNS.match_any].match_all
+    
+This is equivilent to 
+   
+    ("a" | "the") & (WS | (WS & ("big" | "small" | "fierce" | "friendly") & WS)) & ("boy" | "girl" | "dog" | "cat")
+    
+### match? options
+
+The match? method shows above takes a couple of options to globally control the match process:
+
+option | default |  meaning
+------|-----|-----
+ignore_case | false  | When on, the basic regex and string pattern will NOT be case sensitive.
+anchor | false  | When on pattern matching must begin at the first character.  Normally the matcher will keep moving the starting character to the right, until the match suceeds.
+raise_error | false | When on, a match failure will raise Cannonbol::MatchFailed.
+replace_with | nil | When a non-falsy value is supplied, the value will replace the matched portion of the string, and the entire string will be returned.  Normally only the matched portion of the string is returned.
+
+Example of replace with:
+
+    "hello".match?("She said hello!")
+    => hello
+    "hello".match?("She said hello!", replace_with => "goodby")
+    => She said goodby!
+
+### Patterns, Subjects, Cursors, Alternatives, and Backtracking 
+
+A pattern is an object that responds to the match? method.  Cannonbol adds the match? method to Ruby strings, and regexes, and provides a number of _primitive_ patterns.  A pattern can be combined with another pattern using the &, and | operators.  There are also several primitive patterns that take a pattern and create a new pattern.  Here are some example patterns:
+
+    "hello" # matches any string containing hello
+    /\s+/   # matches one or more white space characters
+    "hello" & /\s+/ & "there"  # matches "hello" and "there" seperated by white space
+    "hello" | "goodby"  # matches EITHER "hello" or "there"
+    ARB # a primitive pattern that matches anything (similar to /.*/)
+    ("hello" | "goodby") & ARB & "Fred"  # matches "hello" or "goodby" followed by any characters and finally "Fred"
+
+Patterns are just objects, so they can be assigned to variables:
+
+    greeting = "hello" | "goodby"
+    names = "Fred" | "Suzy"
+    ws = /\s+/
+    greeting & ws & names # matches "hello Fred" or "goodby     Suzy"
+
+The first parameter of the match? method is the subject string.  The subject string is matched left to right driven by the pattern object.  Normally the matcher will attempt to match starting at the first character.  If no match is found, then 
+matching begins again one character to the right.  This continues until a match is made, or there are insufficient characters to make a match.  This behavior can be turned off by specifying `anchor: true` in the match? options hash.
+
+The current position of the matcher in the string is the _cursor_.  The cursor begins at zero and as each character is matched it moves to the right.  If the match fails (and anchor is false) then the match is restarted with the cursor at position 1, etc.
+
+Alternatives are considered left to right as specified in the pattern.  Once an alternative is matched, the matcher moves on to the next part of the match, but it does remember the alternative, and if matching fails at a later component, the matcher will back up and try the next alternative.  For example:
+
+    a_pattern = "a" | "aaa" | "aa"
+    b_pattern = "b" | "aaabb"  | "abbbc"
+    c_pattern = "cc"
+    (a_pattern & b_pattern & c_pattern).match?("aaabbbccc")
+
+* "a" is matched from a_pattern, and then we move to b_pattern.
+* None of the alternatives in b_pattern can match, so we backtrack and try the next alterntive in the a_pattern,
+* "aaa" now matches, and so we move back to the b_pattern and start at the first alternative,
+* "b" now matches, and so we move to the c_pattern,
+* None of the alternatives in the c_pattern can match, so we move back to the b_pattern,
+* None of the remaining alternatives in the b_pattern match, so we move back to the a_pattern,
+* "aa" now matches, and so we move to the b_pattern, which can only match its last alternative, and
+* finally we complete the match!
+
+For a more complete explanation see the [SNOBOL4 manual Chapter 2](http://www.math.bas.bg/bantchev/place/snobol/gpp-2ed.pdf)
+
+Bottom line is the matcher will try every possible option until a match is made or the match fails.
+
+### Basic Primitive Patterns
+
+Cannonbol includes the complete set of SNOBOL4 + SPITBOL primitive patterns and functions.  These are added to the ruby name space via the Object class, and so are available everywhere.
+
+`REM` Match 0 or more characters to the end of the subject string.  
+
+`("the" & REM).match?("he saw the small boy") === "the small boy"`
+
+`ARB` Match 0 or more characters.  ARB first tries to match zero characters, then 1 character, then 2 until the match succeeds. It is roughly equivilent to `\.*\`, except the regex will NOT backtrack like ARB will.
+
+`("the" & ARB & "boy").match?("he saw the small boy running") === "the small boy"`
+
+`LEN(n)` Match any n characters. Equivilent to `\.{n}\`
+
+`POS(x)` Match ONLY if current cursor is at x.  POS(0) is the start of the string.
+
+`(POS(5) & ARB & POS(7)).match?("01234567") === "567"`
+
+`RPOS(x)` Just like POS except measured from the end of the string.  I.e. RPOS(0) is just after the last character.
+
+`("hello" & RPOS(0)).match?("she said hello!")` would fail.
+
+`TAB(x)` Is equivilent to `ARB & POS(x)`.  In otherwords match zero or more characters up to the x'th character. Fails if x < the current cursor.
+
+`RTAB(x)`  You guessed it === `ARB & RPOS(x)`
+
+`ANY(s)` Will match 1 character in s.  So if s = "ABC" it will match A or B or C.  Regexes are generally more useful.
+
+`NOTANY(s)` Will match 1 character as long as its NOT in s.
+
+`SPAN(s)`  Matches 1 or more from s.  Again regexes are generally easier to write.
+
+`BREAK(s)` Matches 0 or more characters until a character in s is hit.
+
+`BREAKX(s)` Woah... like BREAK, but if the match fails, then it will skip the character and try again.  Huh!
+
+`ARBNO(pat)` Match pat zero or more times.
+
+`POS(0) & /\w+/ & ARBNO(/\s*,\s*/ & /\w+/) & /\s*/ & RPOS(0)` will match a list of identifiers separated by commas.
+
+### Delayed Evaluation of Primitive Pattern Parameters
+
+There are several cases where it is useful to delay the evaluation of a primitive pattern arguments until the match is 
+being made, rather than when the pattern is created.
+
+To allow for this all primitive patterns can take a block.  The block is evaluated when the matcher encounters the primitive, and the result of the block is used as the argument to the pattern.
+
+Here is a method that will parse a set of fixed width fields, where the widths are supplied as arguments to the method:
+
+    def parse(s, *widths)
+      fields = []
+      (ARBNO(LEN {widths.shift}.capture? {|field| fields << field}) & RPOS(0)).match?(s)
+      fields
+    end
+
+To really get into the power of delayed evaluation however we need to add two more concepts:
+
+The MATCH primitive, and the capture! (pronounced _capture NOW_) method.  
+
+The capture? (pronounced _capture IF_) method executes when the match has completed successfully.  In contrast the capture! method calls its block as soon as its sub-pattern matches.  Using capture! allows you to pick up values during one phase of the match and then use those values later.  
+
+Meanwhile MATCH takes a pattern as its argument (like ARBNO) but will only match the pattern once.  The power in MATCH is when it is used with a delayed evaluation block.  Together MATCH and capture! allow for patterns that are much more powerful than simple regexes.  For example here is a palindrome matcher:
+    
+     palindrome = MATCH do | ; c|
+       /\W*/ & LEN(1).capture! { |m| c = m } & /\W*/ & ( palindrome | LEN(1) | LEN(0)) & /\W*/ & MATCH { c } 
+     end 
+
+Lets see it again with some comments
+
+    palindrome = MATCH do | ; c | 
+      # By putting the MATCH pattern in a block to be evaluated later we can use palindrome in its definition.
+      # Just to keep things clean and robust we declare c (the character matched) as local to the block.
+      
+      /\W*/ &                          # skip any white space
+      LEN(1).capture! { |m| c = m } &  # grab the next character now and save it in c
+      /\W*/ &                          # skip more white space
+      (                                # now there are three possibilities: 
+        palindrome |                     # there are more characters on the left side of the palindrome OR
+        LEN(1) |                         # we are at the middle ODD character OR
+        LEN(0)                           # the palindrome has an even number of characters
+      ) &                              # now that we have the left half matched, we match the right half
+      /\W*/ &                          # skip any white space and finally
+      MATCH { c }                      # match the same character on the left now on the far right
+      
+    end
+    
+    palindrome.match?('A man, a plan, a canal, Panama!")
+
+Using MATCH to define recursive patterns makes Cannonbol into a full blown BNF parser.  See the example [email address parser](A complete real world example)
+
+### Advanced capture techniques
+
+Both capture? and capture! have a number of useful features.
+
+* They can take a block which is passed the matching substring.
+* As well as the current match, they can pass the current cursor position and the current value of capture variable.
+* They can take a symbol parameter i.e. `capture?(:data)` which will save the value under the name :data.
+* The block can be used to update the capture variable before its aved.
+* They can capture an array of values in a single capture variable.
+
+#### Passing a block to a capture method
+
+This is the most general way of capturing a submatch.  For example
+
+    word = /\W*/ & /\w+/.capture? { |match| words << match } & /\W*/
+
+will shovel each word it matches into the words array.  You could use it like this:
+
+    words = []
+    (ARBNO(word).match?("a big strange, long sentence!")
+
+Using `capture? { |m| puts m }` is handy for debugging your patterns.
+
+#### Current cursor position
+
+The second parameter of the capture block will recieve the current cursor position.   For example
+
+    ("i".capture! { |m, p| puts "i found at #{p-1}"} & RPOS(0)).match("I said hello!", ignore_case: true)
+     => i found at 0
+     => i found at 4
+    
+Notice the use of RPOS(0) which will force the pattern to look at every character in the subject, until the pattern finally fails.  By using capture! (capture NOW) we record every hit, even though the pattern fails in the end.
+
+#### Using capture variables
+
+If the capture methods are supplied with a symbol, then the captured value will be saved in an internal capture variable.  For example:
+
+    some_pattern.capture!(:value)
+    
+would save the string matched by some_pattern into the capture variable called :value.  
+
+There are a couple of ways to retrieve the capture variables:
+
+Any primitive pattern that takes a parameter can use the value of a capture variable.  So for example `LEN(:foo)` means 
+take the current value of the capture variable :foo as the parameter to LEN.
+
+We can use this to clean up the palindrome pattern a little bit:
+
+    palindrome = /\W*/ & LEN(1).capture!(:c) & /\W*/ & ( MATCH{palindrome} | LEN(1) | LEN(0) ) & /\W*/ & MATCH(:c)
+
+Another way to get the capture variables is to interogate the value returned by match?.  The value returned by match? is a subclass of string, that has some extra methods.  One of these is the captured method which gives a hash of all the captured variables.  For example:
+
+    ("dog" | "cat").capture?(:pet).match?("He had a dog named Spot.").captured[:pet]
+    => dog
+
+You can also give a block to the match? method which will be called whether the block passes or not.  For example:
+
+    ("dog" | "cat").capture?(:pet).match?("He had a dog named Spot."){ |match| match.captured[:pet] if match}
+    => dog
+   
+The match? block can also explicitly name any capture variables you need to get the values of.  So for example:
+
+    pet_data = (POS(0) & ARBNO(("big" | "small").capture?(:size) | ("dog" | "cat").capture?(:pet) | LEN(1)) & RPOS(0))
+    pet_data.match?("He has a big dog!") { |m, pet, size| "type of pet: #{pet.upcase}, size: #{size.upcase}"}
+    => type of pet: DOG, size: BIG
+
+If the match? block mentions capture variables that were not assigned in the match they get nil.
+
+#### Initializing capture variables
+
+When used as a parameter to a primitve the capture variable may be given an initial value.   For example:
+    
+    LEN(baz: 12)
+
+would match LEN(12) if :baz had not yet been set.
+
+A second way to initialize (or update capture variables) is to combine capture variables with a capture block like this:
+
+    some_pattern.capture!(:baz) { |match, position, baz| baz || position * 2 } initializes :baz to position * 2
+    
+If a symbol is specified in a capture!, and there is a block, then the symbol will be set to the value returned by the block.
+
+#### Capturing arrays of data
+
+To capture all the words into a capture variable as an array you could do this:
+
+    words = []
+    word = /\W*/ & /\w+/.capture?(:words) { |match| words << match } & /\W*/
+    
+This can be shortened to:
+
+    word = /\W*/ & /\w+/.capture?(:words => []) & /\W*/
+
+This works because anytime there is a 1) capture with a capture variable that is 2) holding an array,  3) that does NOT have a block, capture method will go ahead and shovel the captured value into the capture variable.   Note this behavior can be overriden if needed by including a block.
+
+#### Capture variables and nested patterns
+
+Each time MATCH, or ARBNO is called the current state of any known capture variables are saved, and those values will be restored when the MATCH/ARBNO exits.  If new capture variables are introduced by the nested pattern, these new values will be merged with the existing set of variables.  
+
+More powerful yet is the fact that every match string sent to a capture variable has access to all the values captured so far via the captured method. For example:
+    
+     subject_clause = article & noun.capture!(:subject) 
+     object_clause = article & noun.capture!:object)
+     verb_clause = ...
+     sentence = (subject_clause & verb_clause & object_clause & ".")
+     sentences = ARBNO(sentence.capture?(:sentences => [])) & RPOS(0)
+     sentences.match(file_stream).captured[:sentences].collect(&:captured)
+     => [{:subject => "dog", :object => "man"}, {:subject => "man", :object => "dog} ...]
+
+As each noun is matched, it is captured and saved in :subject or :object.  When the sentence is captured, the match is shoveled away into the :sentences variable.  Because the match value itself responds to the captured method we end up with a all the data collected in a nice array.  
+
+Note that capture! is used for capturing the nouns.  This is cheaper and does not hurt anything since the value of 
+the capture variable will just be overwritten.
+    
+### Advanced PRIMITIVES
+
+There are few more SNOBOL4 + SPITBOL primitives that are included for completeness.
+
+`FENCE` matches the empty string, but will fail if there is an attempt to backtrack through the FENCE.
+`FENCE(pattern)` will attempt to match pattern, but if an attempt is made to backtrack through the FENCE the pattern will fail.
+
+The difference is that FENCE will fail the whole match, but FENCE(pattern) will just fail the subpattern.
+
+`ABORT` unconditionally will exit the match.
+
+`FAIL` will never match anything, and will force the matcher to backtrack and retry the next alternative.
+
+`SUCCEED` will force the match to retry.  The only that gets passed `SUCCEED` is `ABORT`.
+
+These can be used together to do some interesting things.  For example
+
+    pattern = POS(0) & SUCCEED & (FENCE(TAB(n: 1).capture!(:n) { |m, p, n|  puts m; p+1 } | ABORT)) & FAIL
+    pattern.match?("abcd")
+    
+prints
+
+    a
+    ab
+    abc
+    abcd
+    
+The SUCCEED and FAIL primitives keep forcing the matcher to retry.  Eventually the TAB will fail causing the ABORT alternative to execute the matcher.
+
+So it goes like this
+
+    SUCCEED
+    TAB(1)
+    FAIL
+    SUCEED
+    TAB(2)
+    etc...
+    
+The FENCE keeps the matcher from backtracking into the ABORT option too early. Otherwise when the matcher hit fail, it would try different alternatives, and would hit the ABORT.
+
+### A complete real world example
+
+Cannonbol can be used to easily translate the email BNF spec into an email address parser.
+
+    ws             = /\s*/
+    quoted_string  = ws & '"' & ARBNO(NOTANY('"\\') | '\\"' | '\\\n' | '\\\\') & '"' & ws
+    atom           = ws & SPAN("!#$%&'*+-/0123456789=?@ABCDEFGHIJKLMNOPQRSTUVWXYZ^_`abcdefghijklmnopqrstuvwxyz{|}~") & ws
+    word           = (atom | quoted_string)
+    phrase         = word & ARBNO(word)
+    domain_ref     = atom 
+    domain_literal = "[" & /[0-9]+/ & ARBNO(/\.[0-9]+/) & "]"
+    sub_domain     = domain_ref | domain_literal
+    domain         = (sub_domain & ARBNO("." & sub_domain)).capture?(:domain) { |m| m.strip }
+    local_part     = (word & ARBNO("." & word)).capture?(:local_part) { |m| m.strip }
+    addr_spec      = (local_part & "@" & domain)
+    route          = (ws & "@" & domain & ARBNO("@" & domain)).capture?(:route) { |m| m.strip } & ":" 
+    route_addr     = "<" & ((route | "") & addr_spec).capture?(:mailbox) { |m| m.strip } & ">"
+    mailbox        = (addr_spec.capture?(:mailbox) { |m| m.strip } | 
+                     (phrase.capture?(:display_name) { |m| m.strip } & route_addr))  
+    group          = (phrase.capture?(:group_name) { |m| m.strip } & ":" &
+                     (( mailbox.capture?(group_mailboxes: []) & ARBNO("," & mailbox.capture?(:group_mailboxes) ) ) | ws)) & ";"
+    address        = POS(0) & (mailbox | group ) & RPOS(0)
+
+So for example we can even parse an obscure email with groups and routes
+
+    email = 'here is my "big fat \\\n groupen" : someone@catprint.com, Fred Nurph<@sub1.sub2@sub3.sub4:fred.nurph@catprint.com>;'
+    match = address.match?(email)
+    match.captured[:group_mailboxes].first.captured[:mailbox]
+    => someone@catprint.com
+    match.captured[:group_name]
+    => here is my "big fat \\\n groupen
+
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release` to create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+1. Fork it ( https://github.com/[my-github-username]/cannonbol/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request

data/Rakefile

@@ -0,0 +1,3 @@
+require "bundler/gem_tasks"
+
+Dir.glob('tasks/**/*.rake').each(&method(:import))

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "cannonbol"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -euo pipefail
+IFS=$'\n\t'
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/cannonbol.gemspec

@@ -0,0 +1,36 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'cannonbol/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "cannonbol"
+  spec.version       = Cannonbol::VERSION
+  spec.authors       = ["catmando"]
+  spec.email         = ["mitch@catprint.com"]
+
+  if spec.respond_to?(:metadata)
+  end
+
+  spec.summary       = %q{Cannonbol is a ruby dsl for patten matching based on SNOBOL4 and SPITBOL}
+  spec.description   = %q{
+Makes complex patterns easier to read and write!
+Combine regexes, plain strings and powerful new primitive match functions!
+Makes capturing match results easy!
+Allows recursive patterns!
+Complete SNOBOL4 + SPITBOL extensions!
+Based on the well documented, proven SNOBOL4 language!
+Simple syntax looks great alongside ruby!
+                       }
+  spec.homepage      = "https://github.com/catprintlabs/cannonbol"
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.8"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec"
+end

data/lib/cannonbol.rb

@@ -0,0 +1,664 @@
+require "cannonbol/version"
+
+module Cannonbol
+  
+  class MatchFailed < Exception; end
+    
+  class MatchString < String
+    
+    attr_reader :captured
+    attr_reader :match_start
+    attr_reader :match_end
+    
+    def initialize(string, match_start, match_end, captured)
+      @cannonbol_string = string
+      @match_start = match_start
+      @match_end = match_end
+      @captured = captured.dup
+      super(@match_end < 0 ? "" : string[@match_start..@match_end])
+    end 
+    
+    def replace_match_with(s)
+      @cannonbol_string.dup.tap do |new_s| 
+        new_s[@match_start..@match_end] = "" if @match_end >= 0
+        new_s.insert(@match_start, s)
+      end 
+    end
+    
+  end
+  
+  class Needle
+    
+    attr_reader :cursor
+    attr_reader :string
+    attr_accessor :captures
+    attr_accessor :match_failed
+    attr_accessor :ignore_case
+    
+    def initialize(string)
+      @string = string
+    end
+    
+    def thread(pattern, opts = {}, &match_block)
+      @captures = {}
+      anchor = opts[:anchor]
+      raise_error = opts[:raise_error]
+      replace_with = opts[:replace_match_with]
+      ignore_case = opts[:ignore_case]
+      @cursor = -1
+      match = nil
+      begin
+        while !match and !match_failed and @cursor < @string.length-1
+          @cursor += 1
+          @starting_character = nil
+          @success_blocks = []
+          @ignore_case = ignore_case
+          match = pattern._match?(self)
+          break if anchor and !match 
+        end
+      rescue MatchFailed
+      end
+      if match
+        @success_blocks.each(&:call)
+        match = MatchString.new(@string, @starting_character || @cursor, @cursor-1, @captures)
+      else
+        raise MatchFailed if raise_error
+      end
+      if match_block
+        match = match_block.call(*([match] + (match_block.parameters[1..-1] || []).collect { |param| @captures[param[1].to_sym] }))
+      elsif replace_with
+        match = match.replace_match_with(replace_with)
+      end
+      match
+    end
+    
+    def capture(name, value)
+      @captures[name.to_sym] = value if name
+      value
+    end
+      
+       
+    def remaining_string
+      @string[@cursor..-1]
+    end
+     
+    def push(length, &success_block)
+      thread_state = [@starting_character, @cursor, @success_blocks.dup, @ignore_case]
+      @starting_character ||= @cursor
+      @cursor += length
+      @success_blocks << success_block if success_block
+      thread_state
+    end
+      
+    def pull(thread_state)
+      @starting_character, @cursor, @success_blocks, @ignore_case = thread_state if thread_state
+      nil
+    end
+      
+  end
+  
+  module Operators
+        
+    def _match?(needle, *args, &block)
+      return if needle.match_failed
+      __match?(needle, *args, &block)
+    end
+    
+    def match?(s, opts = {}, &match_block)
+      Needle.new(s).thread(self, opts, &match_block)
+    end
+    
+    def |(pattern)
+      Choose.new(self, pattern)
+    end
+  
+    def &(pattern)
+      Concat.new(self, pattern)
+    end
+    
+    def -@
+      CaseSensitiveOff.new(self)
+    end 
+
+    def capture?(opts = {}, &block)
+      OnSuccess.new(self, opts, &block)
+    end
+    
+    def capture!(opts = {},  &block)
+      OnMatch.new(self, opts, &block)
+    end
+    
+  end
+   
+  class Pattern < Array
+    
+    include Operators
+    
+    def to_s
+      "#{self.class.name}[#{self.collect(&:to_s).join(', ')}]"
+    end
+    
+    def __match?(needle)
+      []
+    end
+      
+  end
+  
+  class Choose < Pattern
+    
+    def __match?(needle, i = 0, s = [])
+      while i < self.length
+        s = self[i]._match?(needle, *s)
+        return [i, s] if s
+        s = []
+        i += 1
+      end
+    end
+    
+    def initialize(p1, p2)
+      self << p1 << p2
+    end
+    
+  end
+  
+  class Concat < Pattern
+    
+    def __match?(needle, i = 0, s = [])
+      while i < self.length and i >= 0
+        s[i] = self[i]._match?(needle, *(s[i] || []))
+        i = s[i] ? i+1 : i-1
+      end
+      [i-1, s] if i == self.length
+    end
+    
+    def initialize(p1, p2)
+      self << p1 << p2
+    end
+    
+  end
+  
+  class CaseSensitiveOff < Pattern
+    
+    def initialize(pattern)
+      @pattern = pattern
+    end 
+    
+    def __match?(needle, thread=nil, s=[])
+      needle.pull(thread)
+      thread = needle.push(0)
+      needle.ignore_case = true
+      s = @pattern._match?(needle, *s)
+      return [thread, s] if s
+    end
+    
+  end
+  
+  class OnSuccess < Pattern
+    
+    def initialize(pattern, opts, &block)
+      if opts.class == Hash
+        if opts.first
+          @capture_name = opts.first.first
+          @initial_capture_value = opts.first.last
+        end 
+      else
+        @capture_name = opts
+      end
+      @pattern = pattern
+      @block = block
+    end
+    
+    def __match?(needle, thread_state = nil, starting_cursor = nil, s=[])
+      needle.pull(thread_state)
+      starting_cursor ||= needle.cursor
+      if s = @pattern._match?(needle, *s)
+        ending_cursor = needle.cursor-1
+        push = needle.push(0) do
+          match_string = MatchString.new(needle.string, starting_cursor, ending_cursor, needle.captures)
+          capture_value = @capture_name && (needle.captures.has_key?(@capture_name) ? needle.captures[@capture_name] : @initial_capture_value) 
+          if @block
+            match_string = @block.call(match_string, ending_cursor+1, capture_value) 
+          elsif capture_value.class == Array
+            match_string = capture_value + [match_string]
+          end
+          needle.capture(@capture_name, match_string)
+        end
+        [ push, starting_cursor, s ]
+      end
+    end
+    
+  end
+  
+  class OnMatch < OnSuccess
+    
+    def __match?(needle, starting_cursor = nil, s=[])
+      starting_cursor ||= needle.cursor
+      if s = @pattern._match?(needle, *s)
+        match_string = MatchString.new(needle.string, starting_cursor, needle.cursor-1, needle.captures)
+        capture_value = @capture_name && (needle.captures.has_key?(@capture_name) ? needle.captures[@capture_name] : @initial_capture_value) 
+        match_string = @block.call(match_string, needle.cursor, capture_value) if @block
+        needle.capture(@capture_name, match_string)
+        [starting_cursor, s]
+      end
+    end
+    
+  end
+  
+  class Match < Pattern
+    
+    def initialize(sub_pattern_or_name = nil, &block)
+      if block
+        @block = block
+      elsif sub_pattern_or_name and sub_pattern_or_name.class == Symbol
+        @name = sub_pattern_or_name
+      elsif sub_pattern_or_name and sub_pattern_or_name.respond_to? "_match?"
+        @pattern = sub_pattern_or_name 
+      elsif sub_pattern_or_name and sub_pattern_or_name.respond_to? "to_s"
+        @pattern = sub_pattern_or_name.to_s
+      end
+    end 
+    
+    def __match?(needle, pattern = nil, s = [])
+      pattern ||= if @block
+        @block.call
+      elsif @name
+        needle.captures[@name] || ""
+      else
+        @pattern
+      end
+      existing_captures = needle.captures.dup
+      s = pattern._match?(needle, *s)
+      needle.captures = needle.captures.merge(existing_captures)
+      [pattern, s] if s
+    end 
+    
+  end
+  
+  class Rem < Pattern
+    
+    def __match?(needle, thread_state = nil)
+      if thread_state
+        needle_pull(thread_state)
+      else
+        [needle.push(needle.string.length-needle.cursor)]
+      end 
+    end
+    
+  end
+  
+  class Arb < Pattern
+    
+    def __match?(needle, match_length = 0, thread_state = nil)
+      needle.pull(thread_state)
+      if needle.remaining_string.length >= match_length 
+        thread_state = needle.push(match_length)
+        match_length += 1
+        [match_length, thread_state]
+      end
+    end
+    
+  end
+  
+  class ParameterizedPattern < Pattern
+    
+    def initialize(opts = nil, &block)
+      if opts.class == Hash
+        if opts.first
+          @param_name = opts.first.first
+          @initial_param_value = opts.first.last
+        end 
+      else
+        @initial_param_value = opts
+      end
+      @block = block
+      self << @param << @block
+    end
+    
+    def self.parameter(name, &post_processor)
+      @post_processor = post_processor
+      define_method(name) do |needle|
+        value = (@param_name && needle.captures.has_key?(@param_name)) ? needle.captures[@param_name] : @initial_param_value
+        value = @block.call(value) if @block
+        needle.capture(@param_name, value)        
+        value = post_processor.call(value) if @post_processor
+        value
+      end
+    end
+    
+  end
+  
+  class Len < ParameterizedPattern
+    
+    parameter :len
+    
+    def __match?(needle, thread_state = nil)
+
+      if thread_state
+        needle.pull(thread_state)
+      else
+        len_temp = len(needle)
+        [needle.push(len_temp)] if needle.remaining_string.length >= len_temp
+      end
+      
+    end
+    
+  end
+  
+  class Pos < ParameterizedPattern
+    
+    parameter :pos
+    
+    def __match?(needle, matched = nil)
+      return [true] if needle.cursor == pos(needle) and !matched
+    end 
+    
+  end 
+  
+  class RPos < ParameterizedPattern
+    
+    parameter :pos
+    
+    def __match?(needle, matched = nil)
+      return [true] if needle.string.length-needle.cursor == pos(needle) and !matched
+    end 
+    
+  end
+  
+  class Tab < ParameterizedPattern
+    
+    parameter :pos
+    
+    def __match?(needle, thread_state = nil)
+      
+      if thread_state
+        needle.pull(thread_state)
+      else
+        len = pos(needle) - needle.cursor
+        [needle.push(len)] if len >= 0 and needle.remaining_string.length >= len 
+      end
+    end
+    
+  end
+  
+  class RTab < ParameterizedPattern
+    
+    parameter :pos
+    
+    def __match?(needle, thread_state = nil)
+      if thread_state
+        needle.pull(thread_state)
+      else
+        len = (needle.remaining_string.length - pos(needle)) 
+        [needle.push(len)] if len >= 0 and needle.remaining_string.length >= len
+      end
+    end
+    
+  end
+  
+  class Any < ParameterizedPattern
+    
+    parameter :chars, &:split
+    
+    def __match?(needle, thread_state = nil)
+      if thread_state
+        needle.pull(thread_state)
+      elsif chars(needle).include? needle.remaining_string[0..0]
+        [needle.push(1)]
+      end
+    end
+    
+  end
+  
+  class NotAny < ParameterizedPattern
+    
+    parameter :chars, &:split
+    
+    def __match?(needle, thread_state = nil)
+      if thread_state
+        needle.pull(thread_state)
+      elsif !(chars(needle).include? needle.remaining_string[0..0])
+        [needle.push(1)]
+      end
+    end
+    
+  end  
+  
+  class Span < ParameterizedPattern
+    
+    parameter :chars, &:split
+    
+    def __match?(needle, match_length = nil, thread_state = nil)
+      unless match_length
+        the_chars, match_length = chars(needle), 0
+        while needle.remaining_string.length > match_length and the_chars.include? needle.remaining_string[match_length..match_length]
+          match_length += 1
+        end 
+      end
+      needle.pull(thread_state)
+      if match_length > 0 
+        thread_state = needle.push(match_length)
+        match_length -= 1
+        [match_length, thread_state]
+      end
+    end
+    
+  end
+  
+  class Break < ParameterizedPattern
+    
+    parameter :chars, &:split
+    
+    def __match?(needle, thread_state = nil)
+      if thread_state
+        needle.pull(thread_state)
+      else
+        the_chars, len = chars(needle), 0
+        while needle.remaining_string.length > len and !(the_chars.include? needle.remaining_string[len..len])
+          len += 1
+        end 
+        [needle.push(len)]
+      end 
+    end 
+    
+  end 
+
+  
+  class BreakX < ParameterizedPattern
+    
+    parameter :chars, &:split
+    
+    def __match?(needle, len = 0, thread_state = nil)
+      needle.pull(thread_state)
+      the_chars = chars(needle)
+      while needle.remaining_string.length > len and !(the_chars.include? needle.remaining_string[len..len])
+        len += 1
+      end 
+      [len+1, needle.push(len)] if needle.remaining_string.length >= len
+    end 
+    
+  end   
+  
+  class Arbno < Match
+    
+    def __match?(needle, pattern = nil, s = [[]])
+      return if s.length == 0
+      if pattern
+        existing_captures = needle.captures.dup
+        s[-1] = pattern._match?(needle, *(s.last))
+        s = s[-1] ? s + [[]] : s[0..-2]
+        needle.captures = needle.captures.merge(existing_captures)
+      else
+        if @block
+          pattern = @block.call
+        elsif @name
+          pattern = needle.captures[@name] || ""
+        else
+          pattern = @pattern
+        end
+      end
+      [pattern, s]
+    end 
+    
+  end
+  
+  class FailPat < Pattern
+    
+    def __match?(needle)
+    end 
+    
+  end
+  
+  class Abort < Pattern
+    
+    def __match?(needle)
+      raise MatchFailed
+    end 
+    
+  end
+  
+  class Fence < Match
+    
+    def __match?(needle, on_backtrack = nil)
+      if on_backtrack == :fail_match
+        needle.match_failed = true
+        return nil
+      elsif on_backtrack == :return_nil
+        return nil
+      elsif @block
+        pattern = @block.call
+      elsif @name
+        pattern = needle.captures[@name] || ""
+      elsif @pattern
+        pattern = @pattern
+      else
+        return [:fail_match]
+      end
+      return [:return_nil] if pattern._match?(needle)
+    end 
+    
+  end 
+  
+  class Succeed < Pattern
+    def _match?(needle, thread_state = nil)
+      needle.pull(thread_state)
+      [needle.push(0)]
+    end
+  end
+  
+end
+
+class String
+  
+  include Cannonbol::Operators
+  
+  def __match?(needle, thread_state = nil)
+    if thread_state
+      needle.pull(thread_state)
+    elsif self.length == 0 or 
+          (!needle.ignore_case and needle.remaining_string[0..self.length-1] == self) or 
+          (needle.ignore_case and needle.remaining_string[0..self.length-1].upcase == self.upcase)
+      [needle.push(self.length)]
+    end
+  end
+  
+end
+
+class Regexp
+  
+  include Cannonbol::Operators
+  
+  def __match?(needle, thread_state = nil)
+    @cannonbol_regex ||= Regexp.new("^#{self.source}", self.options | (needle.ignore_case ? Regexp::IGNORECASE : 0) )
+    if thread_state
+      needle.pull(thread_state)
+    elsif m = @cannonbol_regex.match(needle.remaining_string)
+      [needle.push(m[0].length)]
+    end
+  end
+  
+end
+
+module Enumerable
+  
+  def match_any
+    if self.first 
+      self[1..-1].inject(self.first) { |memo, item| memo | item }
+    else
+      FAIL
+    end
+  end 
+  
+  def match_all
+    self.inject("") { |memo, item| memo & item }
+  end 
+  
+end
+  
+
+class Object
+    
+  REM = Cannonbol::Rem.new
+  
+  ARB = Cannonbol::Arb.new
+  
+  FAIL = Cannonbol::FailPat.new
+  
+  ABORT = Cannonbol::Abort.new
+  
+  FENCE = Cannonbol::Fence.new
+  
+  SUCCEED = Cannonbol::Succeed.new
+  
+  def LEN(p={}, &block)
+    Cannonbol::Len.new(p, &block)
+  end
+  
+  def POS(p=nil, &block)
+    Cannonbol::Pos.new(p, &block)
+  end  
+  
+  def RPOS(p=nil, &block)
+    Cannonbol::RPos.new(p, &block)
+  end  
+  
+  def TAB(p=nil, &block)
+    Cannonbol::Tab.new(p, &block)
+  end
+  
+  def RTAB(p=nil, &block)
+    Cannonbol::RTab.new(p, &block)
+  end
+  
+  def ANY(p=nil, &block)
+    Cannonbol::Any.new(p, &block)
+  end 
+  
+  def NOTANY(p=nil, &block)
+    Cannonbol::NotAny.new(p, &block)
+  end
+    
+  def SPAN(p=nil, &block)
+    Cannonbol::Span.new(p, &block)
+  end
+     
+  def BREAK(p=nil, &block)
+    Cannonbol::Break.new(p, &block)
+  end 
+  
+  def BREAKX(p=nil, &block)
+    Cannonbol::BreakX.new(p, &block)
+  end 
+   
+  def MATCH(p=nil, &block)
+    Cannonbol::Match.new(p, &block)
+  end 
+   
+  def ARBNO(p=nil, &block)
+    Cannonbol::Arbno.new(p, &block)
+  end  
+  
+  def FENCE(p=nil, &block)
+    Cannonbol::Fence.new(p, &block)
+  end
+  
+end

data/lib/cannonbol/version.rb

@@ -0,0 +1,3 @@
+module Cannonbol
+  VERSION = "1.0.0"
+end

data/tasks/rspec.rake

@@ -0,0 +1,3 @@
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)

metadata

@@ -0,0 +1,104 @@
+--- !ruby/object:Gem::Specification
+name: cannonbol
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- catmando
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2015-03-17 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.8'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.8'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: "\nMakes complex patterns easier to read and write!\nCombine regexes,
+  plain strings and powerful new primitive match functions!\nMakes capturing match
+  results easy!\nAllows recursive patterns!\nComplete SNOBOL4 + SPITBOL extensions!\nBased
+  on the well documented, proven SNOBOL4 language!\nSimple syntax looks great alongside
+  ruby!\n                       "
+email:
+- mitch@catprint.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".travis.yml"
+- CODE_OF_CONDUCT.md
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/console
+- bin/setup
+- cannonbol.gemspec
+- lib/cannonbol.rb
+- lib/cannonbol/version.rb
+- tasks/rspec.rake
+homepage: https://github.com/catprintlabs/cannonbol
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.2.2
+signing_key: 
+specification_version: 4
+summary: Cannonbol is a ruby dsl for patten matching based on SNOBOL4 and SPITBOL
+test_files: []