birman 0.1.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b986fd1949d2c3755859b6acd8753027417e40a382e18a0bdd02e350133bb1fa
-  data.tar.gz: cd8f04b2b86f234cf915b7b1ad941574f03bfe4f2f1f35fd6488bc94ec27911a
+  metadata.gz: 6e56dfecb27c1584f8aefdb873503084e140abf83d53f2b85ed453ad1e25dbbe
+  data.tar.gz: 35b4ee8725298c3809fbde953f51fca4c550ac207a68a2eade43ad7745e02b15
 SHA512:
-  metadata.gz: 430c9e5d8b26483919f9eb0caaf24c39614752b117e4e61b45faa2919598453a75558a6e928d592055b765563a5fe1a531022ce4311687cc3689bdc6c3f961b2
-  data.tar.gz: 7856edc44f834226c32f014a467de9d89fd5f4e4e840475f3c14b021199ae6de4b62b59520fc5eb9d813e17eb4e521e1e64fc3d74c7db1b49ea983b4ed090d1c
+  metadata.gz: abf8ca7031fe77792fb4a3e0a2c8480358320b2edfeebe0e8da7de0814cf71cbccc5229cf8fb710ec1ddebd49208a9b6f7cf8d9a3b6a066830b57b7271e90827
+  data.tar.gz: bdf4b37b3adcfc42ce7f5eb1bee447c83bbe5cc25ab4795a6478991775f9d913554426abf5c6b1392314f072fced9e7f53b8615ffb4fe5eeb37d0483764ece40

data/CHANGELOG.md

@@ -1,5 +1,33 @@
 ## [Unreleased]
 
+## [0.2.1] - 2026-10-17
+
+- **Fixed:** `rep` and `maybe` over something that can match empty.
+  An empty first match now counts, so `match(/a*/).rep(1)` accepts `""`
+  and `match(/a*/).maybe` gives `""` instead of `nil`. After real
+  matches, an empty one still just ends the repetition. (A plain
+  `.rep` over such a matcher now gives `[""]` on empty input, not `[]`.)
+- **Fixed:** a `label` now replaces the low-level expectations when its
+  matcher succeeds without consuming anything, as it already did when
+  the matcher failed: `str("a").maybe.label("prefix") >> str("b")` on
+  `"x"` reports `expected prefix or "b"`.
+
+## [0.2.0] - 2026-10-17
+
+- **Added:** `one_of` takes a hash mapping literals to the values they
+  produce: `one_of("true" => true, "false" => false, "null" => nil)`.
+- **Added:** `only(x)` inside `seq` makes that part the sequence's
+  whole value; the other parts must still match but are dropped:
+  `seq("[", commit, only(sep_by(value, ",")), "]")`. Its `.as` name, if
+  any, passes outward. Using it twice, mixing it with other `.as`
+  names, or using it outside `seq` raises `ArgumentError`.
+- **Changed:** a bare string is a literal everywhere a matcher goes, not
+  just inside `seq`, `either` and the other helpers. A string used as a
+  rule body (`rule(:semi) { ";" }`) or after `>>`, `|`, `&` or `>` now
+  skips trailing spacing and, if word-like, is a keyword. Before, it
+  matched exactly those characters, so `seq(semi, "x")` failed on
+  `"; x"`. Use `str("...")` for the old exact match.
+
 ## [0.1.0] - 2026-10-17
 
 First release: a small PEG parser library for Ruby 3.2+.

data/README.md

@@ -4,6 +4,9 @@
 
 # Birman
 
+[![Gem Version](https://img.shields.io/gem/v/birman)](https://rubygems.org/gems/birman)
+[![CI](https://github.com/phaedryx/birman/actions/workflows/main.yml/badge.svg)](https://github.com/phaedryx/birman/actions/workflows/main.yml)
+
 A small PEG parser library for Ruby 3.2+: module-based grammars,
 immutable `Data` matchers and AST nodes with source spans, packrat
 memoization, commit points, labels, and `Success`/`Failure` results.
@@ -14,6 +17,12 @@ memoization, commit points, labels, and `Success`/`Failure` results.
 bundle add birman
 ```
 
+or in your Gemfile:
+
+```ruby
+gem "birman", "~> 0.2.1"
+```
+
 ```ruby
 require "birman"
 
@@ -68,7 +77,8 @@ end
 ```
 
 See `examples/calc.rb` for a complete small language with an
-interpreter, and `Grammar.to_ebnf` to print any grammar.
+interpreter, `examples/json.rb` for a full JSON parser (tested against
+`JSON.parse`), and `Grammar.to_ebnf` to print any grammar.
 
 ## Building blocks
 
@@ -76,7 +86,9 @@ interpreter, and `Grammar.to_ebnf` to print any grammar.
 |-------------------------------|------------------------------------------------------|
 | `seq(a, b, c)`                | sequence                                             |
 | `seq(a, commit, b, c)`        | after `a`, the rest must match or parsing stops here |
+| `seq("[", only(x), "]")`      | value is just `x`; the rest must match but is dropped |
 | `either(a, b, c)`             | ordered choice                                       |
+| `one_of("true" => true, ...)` | one of these literals, producing its value (tried in order) |
 | `x.as(:key)`                  | name a part; the sequence's value becomes a hash     |
 | `x.to(Node)`                  | build a node from that hash (or from a single value) |
 | `infix(operand, "+", "-", node: BinOp)` | left-associative binary operators; `BinOp` needs `op`, `left`, `right` fields |
@@ -97,9 +109,11 @@ first: `>>`, `&`, `|`, `>`.
 Declare `ignore :spacing` and whitespace (and comments, with
 `Rules::Comments`) takes care of itself:
 
-- Strings passed to `seq`, `either`, `infix`, `parens` and friends are
-  **literals**. They skip trailing spacing, and word-like ones
-  (`"let"`) are keywords, so `"let"` won't match the start of `letter`.
+- Bare strings are **literals** wherever a matcher goes: in `seq`,
+  `either`, `parens` and friends, as a rule body (`rule(:semi) { ";" }`),
+  or after `>>`, `|`, `&` and `>`. They skip trailing spacing, and
+  word-like ones (`"let"`) are keywords, so `"let"` won't match the
+  start of `letter`.
 - `str("...")` and `match(/.../)` are **raw**: exactly those characters,
   no skipping.
 - Rules for tokens (numbers, names, strings) should be declared with
@@ -175,9 +189,11 @@ has one.
 ## Development
 
 ```sh
-bin/setup          # install dependencies
-bundle exec rake   # tests + rubocop
+bin/setup           # install dependencies
+bundle exec rake    # tests + rubocop
 ruby examples/calc.rb
+ruby examples/json.rb
+ruby bench/json.rb  # vs JSON.parse, Parslet, Treetop, Citrus (installs them)
 ```
 
 ## License

data/lib/birman/combinators.rb

@@ -4,14 +4,17 @@ module Birman
   # Marker for seq: everything after it is committed (see Matcher#>).
   COMMIT = Object.new.tap { |o| def o.inspect = "Birman::COMMIT" }.freeze
 
+  # Marks the part of a seq that becomes its whole value (see #only).
+  Only = Data.define(:matcher)
+
   # Building blocks and parameterized rule factories, available in
   # every grammar. They're plain methods, so a module can override
   # them or add more.
   #
-  # Strings passed to seq, either, infix, one_of, sep_by, between and
-  # parens are literals (see #lit): they skip trailing spacing when
-  # the grammar declares `ignore`, and word-like ones are keywords. Use
-  # str and match for exact, raw matching.
+  # A bare string anywhere a matcher is expected (in seq or either, as
+  # a rule body, after >> or |) is a literal: it skips trailing spacing
+  # when the grammar declares `ignore`, and word-like ones are keywords.
+  # Use str and match for exact, raw matching.
   module Combinators
     # --- Raw matchers ---
 
@@ -26,14 +29,18 @@ module Birman
 
     # --- Literals and the readable forms ---
 
-    # A string literal as a token. "let" becomes a keyword, "=" an
-    # exact string; either way followed by the grammar's ignore rule.
-    # Non-strings pass through unchanged.
-    def lit(x)
-      return Birman.coerce(x) unless x.is_a?(String)
-
-      base = x.match?(/\A\w+\z/) ? keyword(x) : str(x)
-      Lexeme.new(base, ignore_rule, x)
+    # Anything usable as a matcher: strings become literals, regexps
+    # raw patterns, and matchers pass through unchanged.
+    def lit(x) = Birman.coerce(x)
+
+    # The matcher a literal stands for: "let" is a keyword, "=" an exact
+    # string, either way followed by the grammar's ignore rule. Built
+    # once per grammar instance.
+    def literal_matcher(text)
+      (@literal_matchers ||= {})[text] ||= begin
+        base = text.match?(/\A\w+\z/) ? keyword(text) : str(text)
+        Lexeme.new(base, ignore_rule, text)
+      end
     end
 
     # Sequence. Put `commit` where the construct becomes unambiguous:
@@ -42,16 +49,29 @@ module Birman
       commit_at = items.index(COMMIT)
       raise ArgumentError, "commit can't come first in a seq: nothing has matched yet" if commit_at&.zero?
 
-      parts = items.reject { _1.equal?(COMMIT) }.map { lit(_1) }
-      Seq.new(parts, commit_at)
+      parts = items.reject { _1.equal?(COMMIT) }
+      only_at = only_index(parts)
+      Seq.new(parts.map { _1.is_a?(Only) ? _1.matcher : lit(_1) }, commit_at, only_at)
     end
 
     def commit = COMMIT
 
+    # Inside seq, makes this part the seq's whole value; the rest still
+    # has to match but is dropped:
+    #   seq("[", commit, only(sep_by(value, ",")), "]")   # => [...]
+    def only(x) = Only.new(lit(x))
+
     # Ordered choice: the first alternative that matches wins.
     def either(*alts) = alts.size == 1 ? lit(alts.first) : Choice.new(alts.map { lit(_1) })
 
-    def one_of(*strings) = either(*strings)
+    # One of several literals. Given a hash, each literal produces its
+    # value: one_of("true" => true, "null" => nil). They're tried in
+    # order, so put longer ones first ("<=" before "<").
+    def one_of(*choices)
+      return either(*choices) unless choices in [Hash => table]
+
+      either(*table.map { |text, value| lit(text).map { value } })
+    end
 
     # Left-associative binary operators, folded into nodes:
     #   infix(product, "+", "-", node: BinOp)   # BinOp = node(:op, :left, :right)
@@ -70,6 +90,13 @@ module Birman
 
     private
 
+    def only_index(parts)
+      indexes = parts.each_index.select { parts[_1].is_a?(Only) }
+      raise ArgumentError, "a seq can have only one only(...)" if indexes.size > 1
+
+      indexes.first
+    end
+
     def ignore_rule = self.class.respond_to?(:ignore) ? self.class.ignore : nil
   end
 end

data/lib/birman/matchers.rb

@@ -4,8 +4,9 @@ module Birman
   def self.coerce(x)
     case x
     when Matcher then x
-    when String  then Str.new(x)
+    when String  then Lit.new(x)
     when Regexp  then Pat.new(x)
+    when Only    then raise ArgumentError, "only(...) works inside seq, to make one part the seq's value"
     else raise ArgumentError, "can't make a matcher from #{x.inspect}"
     end
   end
@@ -63,10 +64,20 @@ module Birman
 
     private
 
-    def uncommitted_parts(m) = m.is_a?(Seq) && m.commit_at.nil? ? m.parts : [m]
+    def uncommitted_parts(m) = m.is_a?(Seq) && m.commit_at.nil? && m.only_at.nil? ? m.parts : [m]
   end
 
-  # Literal string.
+  # A bare string: a literal (see Combinators#literal_matcher). What a
+  # literal matches depends on the grammar's ignore rule, so the grammar
+  # supplies the real matcher when this one runs.
+  Lit = Data.define(:text) do
+    include Matcher
+
+    def match(ctx, pos) = ctx.grammar.literal_matcher(text).match(ctx, pos)
+    def to_ast = [:str, text]
+  end
+
+  # Exact string, nothing skipped (see Combinators#str).
   Str = Data.define(:str) do
     include Matcher
 
@@ -100,11 +111,17 @@ module Birman
   # Sequence: parts match in order. The value is an array, or a hash
   # of the named parts if any are named (see Matcher#as). Parts at
   # index `commit_at` and beyond are committed: if one fails, parsing stops.
-  Seq = Data.define(:parts, :commit_at) do
+  Seq = Data.define(:parts, :commit_at, :only_at) do
     include Matcher
 
     # Arrays are copied and frozen so a matcher is immutable all the way down.
-    def initialize(parts:, commit_at: nil) = super(parts: parts.dup.freeze, commit_at:)
+    def initialize(parts:, commit_at: nil, only_at: nil)
+      if only_at && parts.each_with_index.any? { |part, i| i != only_at && part.capture_key }
+        raise ArgumentError, "a seq with only(...) can't also name parts with .as"
+      end
+
+      super(parts: parts.dup.freeze, commit_at:, only_at:)
+    end
 
     def >>(other) = with(parts: [*parts, Birman.coerce(other)])
 
@@ -123,6 +140,9 @@ module Birman
 
     def to_ast = [:seq, commit_at, parts.map(&:to_ast)]
 
+    # With only(...), the kept part's name (if any) passes outward.
+    def capture_key = only_at && parts[only_at].capture_key
+
     private
 
     # If the failed part recorded nothing at or past pos (it's quiet,
@@ -134,6 +154,8 @@ module Birman
     end
 
     def shape(values)
+      return values[only_at] if only_at
+
       keys = parts.map(&:capture_key)
       return values if keys.none?
 
@@ -213,7 +235,13 @@ module Birman
       values = []
       while max.nil? || values.size < max
         r = inner.match(ctx, pos) or break
-        break if r.pos == pos # an empty match would loop forever
+        # Matching nothing ends the loop (repeating it would never stop).
+        # It still counts as the first match, so `x.rep(1)` and `x.maybe`
+        # work when x can match empty; after real matches it's just the end.
+        if r.pos == pos
+          values << r.value if values.empty?
+          break
+        end
 
         values << r.value
         pos = r.pos
@@ -269,8 +297,9 @@ module Birman
     def capture_key = inner.capture_key
   end
 
-  # Names a matcher for error messages. If inner fails without getting
-  # past its starting position, the error says "expected <name>"
+  # Names a matcher for error messages. If inner fails (or succeeds
+  # while consuming nothing) without getting past its starting
+  # position, the error says "expected <name>"
   # instead of listing every low-level alternative. If it got further,
   # the deeper (more specific) error is kept.
   Label = Data.define(:inner, :name) do
@@ -278,7 +307,7 @@ module Birman
 
     def match(ctx, pos)
       result, furthest, expected = ctx.isolate(pos) { inner.match(ctx, pos) }
-      if result.nil? && furthest == pos
+      if furthest == pos && (result.nil? || (result.pos == pos && expected.any?))
         ctx.fail_at(pos, name)
       else
         ctx.merge(furthest, expected)

data/lib/birman/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Birman
-  VERSION = "0.1.0"
+  VERSION = "0.2.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: birman
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Tad Thorley