parsanol 3.0.0

data/example/markup/basic.rb.md

@@ -0,0 +1,118 @@
+# Markup Parser - Ruby Implementation
+
+## How to Run
+
+```bash
+cd parsanol-ruby/example/markup
+ruby basic.rb
+```
+
+## Code Walkthrough
+
+### Document Structure
+
+A document is a sequence of blocks:
+
+```ruby
+rule(:document) { block.repeat(1).as(:document) }
+
+rule(:block) {
+  heading |
+  unordered_list |
+  paragraph |
+  blank_line.as(:blank)
+}
+```
+
+Blank lines are captured separately and filtered during transformation.
+
+### Heading Rule
+
+Equal signs define heading level (1-3):
+
+```ruby
+rule(:heading) {
+  (str('=').repeat(1, 3).as(:level) >>
+   space >>
+   heading_content.as(:text) >>
+   newline).as(:heading)
+}
+```
+
+`=` is H1, `==` is H2, `===` is H3.
+
+### Paragraph Rule
+
+Consecutive lines form paragraphs:
+
+```ruby
+rule(:paragraph) {
+  (paragraph_line >> newline).repeat(1).as(:paragraph)
+}
+
+rule(:paragraph_line) {
+  (blank_line.absent? >> (str('=').absent? | space.absent?) >> any).repeat(1)
+}
+```
+
+Paragraph lines don't start with `=` followed by space (which would be a heading).
+
+### List Rule
+
+Hyphen-prefixed items:
+
+```ruby
+rule(:unordered_list) {
+  list_item.repeat(1).as(:unordered_list)
+}
+
+rule(:list_item) {
+  (str('-') >>
+   space >>
+   list_content.as(:text) >>
+   newline).as(:item)
+}
+```
+
+Each item must be on its own line with hyphen and space.
+
+## Output Types
+
+```ruby
+# Document with blocks
+MarkupDocument.new([
+  MarkupHeading.new("=", "Title"),
+  MarkupParagraph.new(["text"]),
+  MarkupList.new([{text: "item"}, {text: "item2"}])
+])
+
+# Heading
+MarkupHeading.new("==", "Section")
+# to_html => "<h2>Section</h2>"
+
+# Paragraph
+MarkupParagraph.new(["Line one", "Line two"])
+# to_html => "<p>Line one Line two</p>"
+
+# List
+MarkupList.new([{text: "First"}, {text: "Second"}])
+# to_html => "<ul>\n<li>First</li>\n<li>Second</li>\n</ul>"
+```
+
+## Design Decisions
+
+### Why Equal Signs for Headings?
+
+Equal signs are visually intuitive and don't conflict with common text. They're also easier to type than `#` on some keyboards.
+
+### Why Filter Blank Lines in Transform?
+
+Blank lines separate blocks but aren't content. Filtering them during transformation keeps the AST clean.
+
+### Why Separate Paragraph Lines?
+
+Keeping lines separate allows joining with spaces during HTML generation, preserving word boundaries across line breaks.
+
+### Why Limit Heading Levels to 3?
+
+This markup language is intentionally simple. Real-world use might extend to 6 levels, matching HTML.

data/example/mathn/basic.rb

@@ -0,0 +1,47 @@
+# Demonstrates that we have a compatibility fix to mathn's weird idea of
+# integer mathematics.
+# Originally contributed to Parslet, ported to Parsanol as an example.
+
+$:.unshift File.dirname(__FILE__) + "/../lib"
+
+require 'parsanol/parslet'
+require 'parsanol/convenience'
+include Parsanol::Parslet
+
+def attempt_parse
+  possible_whitespace = match['\s'].repeat
+
+  cephalopod =
+    str('octopus') |
+    str('squid')
+
+  parenthesized_cephalopod =
+    str('(') >>
+    possible_whitespace >>
+    cephalopod >>
+    possible_whitespace >>
+    str(')')
+
+  parser =
+    possible_whitespace >>
+    parenthesized_cephalopod >>
+    possible_whitespace
+
+  # This parse fails, but that is not the point. When mathn is in the current
+  # ruby environment, it modifies integer division in a way that makes 
+  # parslet loop indefinitely.
+  parser.parse %{(\nsqeed)\n}
+rescue Parsanol::ParseFailed
+end
+
+attempt_parse 
+puts 'it terminates before we require mathn'
+
+puts "requiring mathn now"
+# mathn was deprecated as of Ruby 2.5
+if RUBY_VERSION.gsub(/[^\d]/, '').to_i < 250
+  require 'mathn'
+end
+puts "and trying again (will hang without the fix)"
+attempt_parse # but it doesn't terminate after requiring mathn
+puts "okay!"

data/example/mathn/basic.rb.md

@@ -0,0 +1,96 @@
+# Mathn Compatibility - Ruby Implementation
+
+## How to Run
+
+```bash
+cd parsanol-ruby/example/mathn
+ruby basic.rb
+```
+
+## Code Walkthrough
+
+### The Mathn Problem
+
+Ruby's deprecated `mathn` library changed integer division behavior:
+
+```ruby
+# Without mathn:
+3 / 2  # => 1 (integer division)
+
+# With mathn:
+3 / 2  # => (3/2) (Rational)
+```
+
+This broke Parslet's internal calculations.
+
+### The Grammar
+
+Simple cephalopod matching:
+
+```ruby
+cephalopod =
+  str('octopus') |
+  str('squid')
+
+parenthesized_cephalopod =
+  str('(') >>
+  possible_whitespace >>
+  cephalopod >>
+  possible_whitespace >>
+  str(')')
+```
+
+### The Compatibility Fix
+
+Parsanol includes a fix that works regardless of mathn:
+
+```ruby
+def attempt_parse
+  parser = possible_whitespace >>
+           parenthesized_cephalopod >>
+           possible_whitespace
+
+  # This would hang without the fix
+  parser.parse %{(\nsqeed)\n}
+rescue Parsanol::ParseFailed
+end
+```
+
+### Version Check
+
+The example checks Ruby version:
+
+```ruby
+if RUBY_VERSION.gsub(/[^\d]/, '').to_i < 250
+  require 'mathn'
+end
+```
+
+Mathn was deprecated in Ruby 2.5, so it's only loaded on older Rubies.
+
+## Output Types
+
+```
+it terminates before we require mathn
+requiring mathn now
+and trying again (will hang without the fix)
+okay!
+```
+
+## Design Decisions
+
+### Why This Example?
+
+It documents a historical compatibility issue. Users encountering similar problems can find this reference.
+
+### Why Keep Mathn Support?
+
+Some legacy systems still use mathn. Parsanol aims for broad Ruby version compatibility.
+
+### Ruby-Only Feature
+
+This is purely about Ruby library compatibility. Rust has no equivalent issue.
+
+### Modern Relevance
+
+As of Ruby 2.5+, mathn is deprecated. This example is mostly historical but demonstrates Parslet's robustness.

data/example/mathn/example.json

@@ -0,0 +1,39 @@
+{
+  "id": "mathn",
+  "title": "Mathn Compatibility",
+  "description": "Demonstrate compatibility with Ruby's deprecated mathn library which changed integer division behavior.",
+  "category": "conceptual",
+  "tags": ["mathn", "compatibility", "ruby", "integer-division"],
+  "difficulty": "beginner",
+  "concepts": ["compatibility", "integer division", "Ruby libraries", "legacy support"],
+
+  "motivation": {
+    "why": "Documents handling of Ruby library compatibility issues that affect parsing behavior. The deprecated mathn library changed integer division to return Rationals, which broke Parslet's internal calculations.",
+    "useCases": [
+      "Legacy system support",
+      "Ruby version compatibility",
+      "Handling library conflicts"
+    ]
+  },
+
+  "inputFormat": {
+    "description": "Simple parser input that works regardless of mathn.",
+    "examples": [
+      { "input": "(squid)", "description": "Parenthesized cephalopod", "valid": true },
+      { "input": "(sqeed)", "description": "Invalid - should error gracefully", "valid": false }
+    ]
+  },
+
+  "outputFormat": {
+    "description": "Parse results that work with or without mathn.",
+    "structure": {
+      "result": { "description": "The parse result" }
+    }
+  },
+
+  "rubyOnly": true,
+  "parsletCompatible": true,
+  "implementations": {
+    "ruby": { "basic": "basic.rb" }
+  }
+}

data/example/minilisp/basic.rb

@@ -0,0 +1,94 @@
+# Reproduces [1] using parslet. 
+# [1] http://thingsaaronmade.com/blog/a-quick-intro-to-writing-a-parser-using-treetop.html
+
+$:.unshift File.dirname(__FILE__) + "/../lib"
+
+require 'pp'
+require 'parsanol/parslet'
+require 'parsanol/convenience'
+
+module MiniLisp
+  class Parser < Parsanol::Parser
+    root :expression
+    rule(:expression) {
+      space? >> str('(') >> space? >> body >> str(')') >> space?
+    }
+    
+    rule(:body) {
+      (expression | identifier | float | integer | string).repeat.as(:exp)
+    }
+    
+    rule(:space) {
+      match('\s').repeat(1)
+    }
+    rule(:space?) {
+      space.maybe
+    }
+    
+    rule(:identifier) { 
+      (match('[a-zA-Z=*]') >> match('[a-zA-Z=*_]').repeat).as(:identifier) >> space?
+    }
+    
+    rule(:float) { 
+      (
+        integer >> (
+          str('.') >> match('[0-9]').repeat(1) |
+          str('e') >> match('[0-9]').repeat(1)
+        ).as(:e)
+      ).as(:float) >> space?
+    }
+    
+    rule(:integer) {
+      ((str('+') | str('-')).maybe >> match("[0-9]").repeat(1)).as(:integer) >> space?
+    }
+    
+    rule(:string) {
+      str('"') >> (
+        str('\\') >> any |
+        str('"').absent? >> any 
+      ).repeat.as(:string) >> str('"') >> space?
+    }
+  end
+  
+  class Transform
+    include Parsanol::Parslet
+    
+    attr_reader :t
+    def initialize
+      @t = Parsanol::Transform.new
+      
+      # To understand these, take a look at what comes out of the parser. 
+      t.rule(:identifier => simple(:ident)) { ident.to_sym }
+        
+      t.rule(:string => simple(:str))       { str }
+        
+      t.rule(:integer => simple(:int))      { Integer(int) }
+        
+      t.rule(:float=>{:integer=> simple(:a), :e=> simple(:b)}) { Float(a + b) }
+        
+      t.rule(:exp => subtree(:exp))         { exp }
+    end
+    
+    def do(tree)
+      t.apply(tree)
+    end
+  end
+end
+
+parser = MiniLisp::Parser.new
+transform = MiniLisp::Transform.new
+
+result = parser.parse_with_debug %Q{
+  (define test (lambda ()
+    (begin
+      (display "something")
+      (display 1)
+      (display 3.08))))
+}
+
+# Transform the result
+pp transform.do(result) if result
+
+# Thereby reducing it to the earlier problem: 
+# http://github.com/kschiess/toylisp
+

data/example/minilisp/basic.rb.md

@@ -0,0 +1,133 @@
+# Mini Lisp Parser - Ruby Implementation
+
+## How to Run
+
+```bash
+cd parsanol-ruby/example/minilisp
+ruby basic.rb
+```
+
+## Code Walkthrough
+
+### Expression Rule
+
+S-expressions are recursively defined:
+
+```ruby
+rule(:expression) {
+  space? >> str('(') >> space? >> body >> str(')') >> space?
+}
+```
+
+Whitespace is optional around parentheses for flexible formatting.
+
+### Body Rule
+
+Body contains multiple expressions:
+
+```ruby
+rule(:body) {
+  (expression | identifier | float | integer | string).repeat.as(:exp)
+}
+```
+
+The repeat allows empty lists `()` and nested structures.
+
+### Identifier Rule
+
+Identifiers allow operator characters:
+
+```ruby
+rule(:identifier) {
+  (match('[a-zA-Z=*]') >> match('[a-zA-Z=*_]').repeat).as(:identifier) >> space?
+}
+```
+
+`=` and `*` are valid identifier chars for Lisp operators like `=` and `*`.
+
+### Float Rule
+
+Floats have decimal or exponent parts:
+
+```ruby
+rule(:float) {
+  (
+    integer >> (
+      str('.') >> match('[0-9]').repeat(1) |
+      str('e') >> match('[0-9]').repeat(1)
+    ).as(:e)
+  ).as(:float) >> space?
+}
+```
+
+Captures the integer part and the exponent/fraction separately.
+
+### String Rule
+
+Strings handle escape sequences:
+
+```ruby
+rule(:string) {
+  str('"') >> (
+    str('\\') >> any |
+    str('"').absent? >> any
+  ).repeat.as(:string) >> str('"') >> space?
+}
+```
+
+`str('\\') >> any` handles any escaped character including `\"`.
+
+### Transform Class
+
+Transforms convert parse trees to Ruby objects:
+
+```ruby
+class Transform
+  t.rule(:identifier => simple(:ident)) { ident.to_sym }
+  t.rule(:string => simple(:str))       { str }
+  t.rule(:integer => simple(:int))      { Integer(int) }
+  t.rule(:float=>{:integer=> simple(:a), :e=> simple(:b)}) { Float(a + b) }
+  t.rule(:exp => subtree(:exp))         { exp }
+end
+```
+
+`simple(:x)` matches single values; `subtree(:x)` matches nested structures.
+
+## Output Types
+
+```ruby
+# Parse tree
+{:exp=>[
+  {:identifier=>"+"@s},
+  {:integer=>"1"@s},
+  {:integer=>"2"@s}
+]}
+
+# After transform
+[:+, 1, 2]
+
+# Nested expression
+[:define, :test, [:lambda, [], [:begin,
+  [:display, "something"],
+  [:display, 1],
+  [:display, 3.08]
+]]]
+```
+
+## Design Decisions
+
+### Why Symbol for Identifiers?
+
+Ruby symbols are immutable and efficient for identifiers. They're commonly used for code representation.
+
+### Why Float Assembly in Transform?
+
+The float rule captures integer and exponent parts separately. The transform combines them into a Ruby Float.
+
+### Why subtree for Expressions?
+
+`subtree(:exp)` recursively transforms nested lists. This handles arbitrary nesting depth automatically.
+
+### Why Separate Parser and Transform Classes?
+
+Separation keeps grammar definition clean. The transform can evolve independently of parsing rules.

data/example/modularity/basic.rb

@@ -0,0 +1,47 @@
+$:.unshift File.dirname(__FILE__) + "/../lib"
+
+require 'pp'
+require "parsanol/parslet"
+
+# Demonstrates modular parsers, split out over many classes. Please look at
+# ip_address.rb as well.
+
+module ALanguage
+  include Parsanol::Parslet
+    
+  # Parslet rules are really a special kind of method. Mix them into your
+  # classes!
+  rule(:a_language) { str('aaa') }
+end
+
+# Parslet parsers are parslet atoms as well. Create an instance and chain them
+# to your other rules. 
+#
+class BLanguage < Parsanol::Parser
+  root :blang
+  
+  rule(:blang) { str('bbb') }
+end
+
+# Parslet atoms are really Ruby values, pass them around. 
+c_language = Parsanol.str('ccc')
+  
+class Language < Parsanol::Parser
+  def initialize(c_language)
+    @c_language = c_language
+    super()
+  end
+  
+  root :root
+
+  include ALanguage
+  
+  rule(:root) { str('a(') >> a_language >> str(')') >> space |
+                str('b(') >> BLanguage.new >> str(')') >> space | 
+                str('c(') >> @c_language >> str(')') >> space }
+  rule(:space) { str(' ').maybe }
+end
+
+Language.new(c_language).parse('a(aaa)')
+Language.new(c_language).parse('b(bbb)')
+Language.new(c_language).parse('c(ccc)')