rtoon 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 89392ee400d56b09518657851bd1f69b63c8ca50f9376da702a541cc69eaefae
+  data.tar.gz: 96c0f189e8dfa1c559dd8b69e58ad351fba7f5c7b92c1c70f42f8432d70c5344
+SHA512:
+  metadata.gz: c6a0a8e40b860a2b19b3b4060b4f51e1fc430ffc41e3b24dca5907ed16fc9cfcf056763e4184a640104ee61ec7da4b14c0f00e81dcdbabbcfc8e12d04d642a07
+  data.tar.gz: 38033867f87e4c304e8047d82ed7fb2b94ac4fafed741ed571a29e500c4d1e7eb90a55e69cb66d4639914374a234a1d07237fca2ceb9d120e071b118fd21c1e3

data/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,298 @@
+# TOON Parser - Token Object Oriented Notation
+
+A Ruby gem for parsing Token Object Oriented Notation (TOON), a tabular, schema-based data format with indentation-based structure.
+
+## What is TOON?
+
+TOON is a data serialization format that combines:
+- **Schema declarations** with field definitions
+- **Tabular data** for compact representation
+- **Indentation-based nesting** for hierarchy
+- **Array size hints** for structure clarity
+
+### Example
+
+```toon
+items[1]{users,status}:
+  users[2]{id,name}:
+    1,Ada
+    2,Bob
+  status: active
+```
+
+This parses to:
+
+```ruby
+{
+  "items" => [
+    {
+      "users" => [
+        { "id" => "1", "name" => "Ada" },
+        { "id" => "2", "name" => "Bob" }
+      ],
+      "status" => "active"
+    }
+  ]
+}
+```
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'rtoon'
+```
+
+Or install it yourself:
+
+```bash
+gem install rtoon
+```
+
+## Usage
+
+### Basic Parsing
+
+```ruby
+require 'rtoon'
+
+toon_string = <<~TOON
+  users[2]{id,name}:
+    1,Ada
+    2,Bob
+TOON
+
+result = Rtoon.parse(toon_string)
+# => {"users" => [{"id" => "1", "name" => "Ada"}, {"id" => "2", "name" => "Bob"}]}
+```
+
+### Schema Declarations
+
+Define data structures with schemas:
+
+```toon
+products[3]{id,name,price}:
+  1,Widget,100
+  2,Gadget,200
+  3,Gizmo,300
+```
+
+### Nested Structures
+
+Use indentation for nesting:
+
+```toon
+company[1]{depts,location}:
+  depts[2]{name,size}:
+    engineering,50
+    sales,20
+  location: NYC
+```
+
+### Field Assignments
+
+Simple key-value pairs:
+
+```toon
+name: John
+age: 30
+active: true
+```
+
+## TOON Syntax
+
+### Schema Declaration
+
+```
+identifier[size]{field1,field2,...}:
+```
+
+- **identifier**: Name of the data structure
+- **[size]**: Optional array size hint (can be any number)
+- **{fields}**: Comma-separated list of field names
+- **:** Indicates content follows on indented lines
+
+### Data Rows
+
+```
+value1,value2,value3
+```
+
+Comma-separated values matching the schema fields in order.
+
+### Indentation
+
+- Use **2 spaces** for each nesting level
+- Indentation defines the structure hierarchy
+- Empty lines are ignored
+
+### Complete Example
+
+```toon
+database[1]{tables,version}:
+  tables[3]{name,records,indexed}:
+    users,1000,yes
+    posts,5000,yes
+    comments,20000,no
+  version: 3
+```
+
+Parses to:
+
+```ruby
+{
+  "database" => [
+    {
+      "tables" => [
+        { "name" => "users", "records" => "1000", "indexed" => "yes" },
+        { "name" => "posts", "records" => "5000", "indexed" => "yes" },
+        { "name" => "comments", "records" => "20000", "indexed" => "no" }
+      ],
+      "version" => "3"
+    }
+  ]
+}
+```
+
+## API Reference
+
+### `Rtoon.parse(string)`
+
+Parses a TOON string and returns a Ruby hash/array structure.
+
+**Parameters:**
+- `string` (String): TOON formatted string
+
+**Returns:**
+- Hash or Array with parsed data
+
+**Raises:**
+- `RtoonParser::ParseError`: If the string contains invalid TOON syntax
+
+### `Rtoon.decode(string)`
+
+Alias for `Rtoon.parse(string)`.
+
+## Features
+
+✅ Schema-based declarations
+✅ Tabular data rows
+✅ Indentation-based nesting
+✅ Array size hints
+✅ Field assignments
+✅ Multi-level nesting
+✅ Empty line handling
+
+## Grammar
+
+TOON uses a context-free grammar parsed with Racc (Ruby parser generator):
+
+- **Statements**: Schema blocks or field assignments
+- **Schema Block**: Header + indented content
+- **Schema Header**: `name[size]{fields}:`
+- **Data Rows**: Comma-separated values
+- **Field Assignment**: `name: value`
+- **Indentation**: INDENT/DEDENT tokens (2 spaces)
+
+## Development
+
+### Prerequisites
+
+```bash
+# Ruby 2.7 or higher
+ruby --version
+
+# Racc (included with Ruby)
+ruby -e "require 'racc/parser'; puts 'Racc available'"
+```
+
+### Building from Source
+
+```bash
+# Clone or extract the gem
+cd rtoon
+
+# Compile grammar
+racc -o lib/rtoon.tab.rb lib/rtoon.y
+
+# Run tests
+ruby test/toon_test.rb
+
+# Build gem
+gem build rtoon.gemspec
+```
+
+### Running Tests
+
+```bash
+ruby test/toon_test.rb
+```
+
+Expected output:
+```
+11 runs, 39 assertions, 0 failures, 0 errors, 0 skips
+```
+
+## Architecture
+
+```
+TOON String
+    ↓
+RtoonLexer (tokenization + indentation tracking)
+    ↓
+RtoonParser (Racc-generated parser)
+    ↓
+Ruby Hash/Array
+```
+
+### Components
+
+- **lib/rtoon/lexer.rb**: Indentation-aware lexer
+- **lib/rtoon/parser.y**: Racc grammar definition
+- **lib/rtoon/parser.tab.rb**: Generated parser (don't edit!)
+- **lib/rtoon/encoder.rb**: TOON encoder (Ruby → TOON)
+- **lib/rtoon.rb**: Main interface
+
+## Use Cases
+
+TOON is ideal for:
+
+- **Configuration files** with tabular data
+- **Database seeds** with schemas and rows
+- **API responses** with structured tables
+- **Data exports** in readable format
+- **Test fixtures** with clear structure
+
+## Limitations
+
+- Values are currently returned as strings (no automatic type conversion)
+- No support for nested arrays in data rows
+- No string escaping (commas in values not supported)
+- Numbers are treated as identifiers/strings
+
+## Future Enhancements
+
+- Type inference/conversion (numbers, booleans)
+- String literals with quotes
+- Escape sequences for commas in values
+- Comments support
+- Encoder (Ruby → TOON)
+
+## Contributing
+
+1. Fork the repository
+2. Create your feature branch
+3. Add tests for your changes
+4. Ensure all tests pass
+5. Submit a pull request
+
+## License
+
+MIT License - See LICENSE.txt
+
+## Resources
+
+- Racc: https://github.com/ruby/racc
+- Parser theory: https://en.wikipedia.org/wiki/Parsing
+- Indentation-based parsing: https://en.wikipedia.org/wiki/Off-side_rule

data/TOON_SPEC.md

@@ -0,0 +1,61 @@
+# TOON Format Specification
+
+## Overview
+Token Object Oriented Notation (TOON) is a tabular, schema-based data format with indentation-based structure.
+
+## Syntax Elements
+
+### Schema Declaration
+```
+identifier[size]{field1,field2,...}:
+```
+
+- **identifier**: Name of the data structure
+- **[size]**: Optional array size hint
+- **{fields}**: Comma-separated list of field names
+- **:** Indicates content follows
+
+### Data Rows
+```
+value1,value2,value3
+```
+
+Comma-separated values corresponding to the schema fields.
+
+### Nesting
+Uses indentation (2 spaces) to indicate nested structures.
+
+### Example
+```
+items[1]{users,status}:
+  users[2]{id,name}:
+    1,Ada
+    2,Bob
+  status: active
+```
+
+## Parsed Structure
+The above should parse to:
+```ruby
+{
+  "items" => [
+    {
+      "users" => [
+        { "id" => "1", "name" => "Ada" },
+        { "id" => "2", "name" => "Bob" }
+      ],
+      "status" => "active"
+    }
+  ]
+}
+```
+
+## Grammar Rules
+
+1. **Top-level**: Schema declaration or field assignment
+2. **Schema**: `name[size]{fields}:` followed by indented content
+3. **Field**: `name: value` (simple assignment)
+4. **Data rows**: Comma-separated values matching schema fields
+5. **Indentation**: 2 spaces per level
+6. **Arrays**: Created from schema declarations with size hint
+7. **Objects**: Created from schema fields

data/lib/rtoon/encoder.rb

@@ -0,0 +1,68 @@
+module Rtoon
+  class Encoder
+    def self.encode(value, indent: 0, key: nil, top_level: true)
+      case value
+      when Hash
+        encode_hash(value, indent: indent, key: key, top_level: top_level)
+      when Array
+        encode_array(value, indent: indent, key: key, top_level: top_level)
+      else
+        "#{value}"
+      end
+    end
+
+    def self.encode_hash(hash, indent:, key:, top_level:)
+      lines = []
+      if key
+        subkeys = hash.keys
+        lines << ("  " * indent + "#{key}{#{subkeys.join(',')}}:")
+        indent += 1
+      end
+
+      hash.each do |k, v|
+        if v.is_a?(Hash)
+          lines << encode_hash(v, indent: indent, key: k, top_level: false)
+        elsif v.is_a?(Array)
+          lines << encode_array(v, indent: indent, key: k, top_level: false)
+        else
+          lines << ("  " * indent + "#{k}: #{v}")
+        end
+      end
+
+      lines.join("\n") + (top_level ? "\n" : "")
+    end
+
+    def self.encode_array(array, indent:, key:, top_level:)
+      return "" if array.empty?
+
+      # Check if it's an array of hashes that themselves contain nested structures
+      if array.first.is_a?(Hash)
+        # If all elements are flat hashes (no arrays inside), compress to CSV form
+        if array.all? { |h| h.values.all? { |v| !v.is_a?(Array) && !v.is_a?(Hash) } }
+          element_keys = array.first.keys
+          lines = []
+          lines << ("  " * indent + "#{key}[#{array.size}]{#{element_keys.join(',')}}:")
+          array.each do |element|
+            row = element.values.join(',')
+            lines << ("  " * (indent + 1) + row)
+          end
+          return lines.join("\n")
+        else
+          # Otherwise, treat each element recursively
+          lines = []
+          lines << ("  " * indent + "#{key}[#{array.size}]{#{array.first.keys.join(',')}}:")
+          array.each do |element|
+            lines << encode_hash(element, indent: indent + 1, key: nil, top_level: false)
+          end
+          return lines.join("\n")
+        end
+      else
+        # Array of primitives
+        lines = []
+        lines << ("  " * indent + "#{key}[#{array.size}]:")
+        array.each { |v| lines << ("  " * (indent + 1) + v.to_s) }
+        lines.join("\n")
+      end
+    end
+  end
+end

data/lib/rtoon/lexer.rb

@@ -0,0 +1,136 @@
+module Rtoon
+  class Lexer
+    attr_reader :line_number, :indent_stack
+
+    def initialize(input)
+      @input = input
+      @lines = input.split("\n")
+      @line_index = 0
+      @pos = 0
+      @current_line = @lines[@line_index] || ""
+      @indent_stack = [0]
+      @pending_dedents = 0
+      @line_number = 1
+    end
+
+    def next_token
+      # Handle pending dedents first
+      if @pending_dedents > 0
+        @pending_dedents -= 1
+        @indent_stack.pop
+        return [:DEDENT, nil]
+      end
+
+      # Skip empty lines
+      while @current_line && @current_line.strip.empty?
+        advance_line
+        return [false, false] if @current_line.nil?
+      end
+
+      return [false, false] if @current_line.nil?
+
+      # Check indentation at start of line
+      if @pos == 0
+        indent = @current_line[/^\s*/].length
+        current_indent = @indent_stack.last
+
+        if indent > current_indent
+          @indent_stack.push(indent)
+          @pos = indent
+          return [:INDENT, nil]
+        elsif indent < current_indent
+          # Calculate how many dedents needed
+          while @indent_stack.length > 1 && @indent_stack[-1] > indent
+            @pending_dedents += 1
+            if @indent_stack[-2] == indent
+              break
+            end
+          end
+
+          if @pending_dedents > 0
+            @pending_dedents -= 1
+            @indent_stack.pop
+            @pos = indent
+            return [:DEDENT, nil]
+          end
+        end
+
+        @pos = indent
+      end
+
+      # Skip whitespace (but not newlines)
+      while @pos < @current_line.length && @current_line[@pos] == ' '
+        @pos += 1
+      end
+
+      # End of line
+      if @pos >= @current_line.length
+        advance_line
+        return [:NEWLINE, "\n"]
+      end
+
+      char = @current_line[@pos]
+
+      case char
+      when '['
+        @pos += 1
+        return [:LBRACKET, '[']
+      when ']'
+        @pos += 1
+        return [:RBRACKET, ']']
+      when '{'
+        @pos += 1
+        return [:LBRACE, '{']
+      when '}'
+        @pos += 1
+        return [:RBRACE, '}']
+      when ':'
+        @pos += 1
+        return [:COLON, ':']
+      when ','
+        @pos += 1
+        return [:COMMA, ',']
+      when '0'..'9'
+        return [:NUMBER, scan_number]
+      when 'a'..'z', 'A'..'Z', '_'
+        return [:IDENTIFIER, scan_identifier]
+      else
+        raise "Unexpected character: #{char.inspect} at line #{@line_number}, position #{@pos}"
+      end
+    end
+
+    private
+
+    def advance_line
+      @line_index += 1
+      @line_number += 1
+      if @line_index < @lines.length
+        @current_line = @lines[@line_index]
+        @pos = 0
+      else
+        @current_line = nil
+      end
+    end
+
+    def scan_number
+      start = @pos
+      # Scan digits
+      @pos += 1 while @pos < @current_line.length && @current_line[@pos].match?(/[0-9]/)
+
+      # Check for decimal point
+      if @pos < @current_line.length && @current_line[@pos] == '.'
+        @pos += 1
+        # Scan fractional part
+        @pos += 1 while @pos < @current_line.length && @current_line[@pos].match?(/[0-9]/)
+      end
+
+      @current_line[start...@pos]
+    end
+
+    def scan_identifier
+      start = @pos
+      @pos += 1 while @pos < @current_line.length && @current_line[@pos].match?(/[a-zA-Z0-9_.]/)
+      @current_line[start...@pos]
+    end
+  end
+end

data/lib/rtoon/parser.tab.rb

@@ -0,0 +1,391 @@
+#
+# DO NOT MODIFY!!!!
+# This file is automatically generated by Racc 1.8.1
+# from Racc grammar file "parser.y".
+#
+
+require 'racc/parser.rb'
+
+require_relative 'lexer'
+
+module Rtoon
+  class Parser < Racc::Parser
+
+module_eval(<<'...end parser.y/module_eval...', 'parser.y', 74)
+
+def parse(str)
+  @lexer = Lexer.new(str)
+  do_parse
+end
+
+def next_token
+  @lexer.next_token
+end
+
+def on_error(token_id, value, value_stack)
+  line = @lexer.line_number
+  raise ParseError, "Parse error at line #{line}: unexpected token #{value.inspect}"
+end
+
+def process_statements(statements)
+  result = {}
+  current_schema = nil
+  data_rows = []
+
+  statements.compact.each do |stmt|
+    case stmt[:type]
+    when :schema
+      # First, finalize any pending schema with data rows
+      if current_schema && !data_rows.empty?
+        result[current_schema[:name]] = build_array_from_rows(current_schema, data_rows)
+        data_rows = []
+      end
+
+      current_schema = stmt[:header]
+
+      # Process nested content
+      nested_result = process_statements(stmt[:content])
+
+      # Check if there are data rows in the nested content
+      nested_data_rows = stmt[:content].select { |s| s && s[:type] == :data_row }
+
+      if !nested_data_rows.empty?
+        # This schema has data rows
+        result[current_schema[:name]] = build_array_from_rows(current_schema, nested_data_rows)
+        current_schema = nil
+      elsif !nested_result.empty?
+        # This schema has nested schemas
+        if current_schema[:size]
+          result[current_schema[:name]] = [nested_result]
+        else
+          result[current_schema[:name]] = nested_result
+        end
+        current_schema = nil
+      end
+
+    when :field
+      result[stmt[:name]] = stmt[:value]
+
+    when :data_row
+      data_rows << stmt[:values]
+    end
+  end
+
+  # Finalize any remaining schema
+  if current_schema && !data_rows.empty?
+    result[current_schema[:name]] = build_array_from_rows(current_schema, data_rows)
+  end
+
+  result
+end
+
+def build_array_from_rows(schema, data_row_statements)
+  fields = schema[:fields]
+
+  data_row_statements.map do |row_stmt|
+    values = row_stmt[:values]
+    obj = {}
+    fields.each_with_index do |field, idx|
+      obj[field] = values[idx] if idx < values.length
+    end
+    obj
+  end
+end
+
+class ParseError < StandardError; end
+...end parser.y/module_eval...
+##### State transition tables begin ###
+
+racc_action_table = [
+     8,    12,     8,    12,     8,    12,     8,    12,    13,    16,
+     6,    17,     6,    18,     6,    31,     6,    32,    38,    15,
+    32,    26,    12,    26,    12,    19,    20,    21,    22,    24,
+    30,    33,    34,    35,    36,    24,    39 ]
+
+racc_action_check = [
+     0,     0,     2,     2,    21,    21,    28,    28,     1,     8,
+     0,     8,     2,     8,    21,    23,    28,    23,    37,     7,
+    37,    18,    18,    19,    19,    10,    13,    15,    16,    17,
+    22,    29,    30,    31,    32,    34,    38 ]
+
+racc_action_pointer = [
+    -2,     8,     0,   nil,   nil,   nil,   nil,     7,     5,   nil,
+    16,   nil,   nil,    26,   nil,    17,    25,    27,    19,    21,
+   nil,     2,    25,     8,   nil,   nil,   nil,   nil,     4,    20,
+    26,    25,    32,   nil,    33,   nil,   nil,    11,    28,   nil ]
+
+racc_action_default = [
+    -2,   -22,    -1,    -3,    -5,    -6,    -7,   -22,   -20,   -16,
+   -17,   -18,   -21,   -22,    -4,   -22,   -22,   -22,   -22,   -22,
+    40,   -22,   -22,   -22,   -12,   -15,   -20,   -19,   -14,    -9,
+   -22,   -22,   -22,    -8,   -22,   -11,   -13,   -22,   -22,   -10 ]
+
+racc_goto_table = [
+    14,     2,    23,    25,    27,     1,    29,   nil,   nil,   nil,
+   nil,   nil,   nil,   nil,   nil,   nil,   nil,   nil,   nil,    37,
+   nil,   nil,    28,   nil,   nil,   nil,    14 ]
+
+racc_goto_check = [
+     3,     2,     8,     9,     9,     1,     7,   nil,   nil,   nil,
+   nil,   nil,   nil,   nil,   nil,   nil,   nil,   nil,   nil,     8,
+   nil,   nil,     2,   nil,   nil,   nil,     3 ]
+
+racc_goto_pointer = [
+   nil,     5,     1,    -2,   nil,   nil,   nil,   -15,   -15,   -15,
+   nil,   nil ]
+
+racc_goto_default = [
+   nil,   nil,   nil,     3,     4,     5,     7,   nil,   nil,    11,
+     9,    10 ]
+
+racc_reduce_table = [
+  0, 0, :racc_error,
+  1, 14, :_reduce_1,
+  0, 14, :_reduce_2,
+  1, 15, :_reduce_3,
+  2, 15, :_reduce_4,
+  1, 16, :_reduce_5,
+  1, 16, :_reduce_6,
+  1, 16, :_reduce_7,
+  5, 17, :_reduce_8,
+  4, 17, :_reduce_9,
+  8, 19, :_reduce_10,
+  5, 19, :_reduce_11,
+  1, 21, :_reduce_12,
+  3, 21, :_reduce_13,
+  1, 20, :_reduce_14,
+  3, 18, :_reduce_15,
+  1, 18, :_reduce_16,
+  1, 23, :_reduce_17,
+  1, 24, :_reduce_18,
+  3, 24, :_reduce_19,
+  1, 22, :_reduce_20,
+  1, 22, :_reduce_21 ]
+
+racc_reduce_n = 22
+
+racc_shift_n = 40
+
+racc_token_table = {
+  false => 0,
+  :error => 1,
+  :IDENTIFIER => 2,
+  :NUMBER => 3,
+  :LBRACKET => 4,
+  :RBRACKET => 5,
+  :LBRACE => 6,
+  :RBRACE => 7,
+  :COLON => 8,
+  :COMMA => 9,
+  :INDENT => 10,
+  :DEDENT => 11,
+  :NEWLINE => 12 }
+
+racc_nt_base = 13
+
+racc_use_result_var = true
+
+Racc_arg = [
+  racc_action_table,
+  racc_action_check,
+  racc_action_default,
+  racc_action_pointer,
+  racc_goto_table,
+  racc_goto_check,
+  racc_goto_default,
+  racc_goto_pointer,
+  racc_nt_base,
+  racc_reduce_table,
+  racc_token_table,
+  racc_shift_n,
+  racc_reduce_n,
+  racc_use_result_var ]
+Ractor.make_shareable(Racc_arg) if defined?(Ractor)
+
+Racc_token_to_s_table = [
+  "$end",
+  "error",
+  "IDENTIFIER",
+  "NUMBER",
+  "LBRACKET",
+  "RBRACKET",
+  "LBRACE",
+  "RBRACE",
+  "COLON",
+  "COMMA",
+  "INDENT",
+  "DEDENT",
+  "NEWLINE",
+  "$start",
+  "document",
+  "statements",
+  "statement",
+  "schema_block",
+  "field_assignment",
+  "schema_header",
+  "block_content",
+  "field_list",
+  "value",
+  "data_row",
+  "value_list" ]
+Ractor.make_shareable(Racc_token_to_s_table) if defined?(Ractor)
+
+Racc_debug_parser = false
+
+##### State transition tables end #####
+
+# reduce 0 omitted
+
+module_eval(<<'.,.,', 'parser.y', 10)
+  def _reduce_1(val, _values, result)
+     result = process_statements(val[0])
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 11)
+  def _reduce_2(val, _values, result)
+     result = {}
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 15)
+  def _reduce_3(val, _values, result)
+     result = [val[0]]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 16)
+  def _reduce_4(val, _values, result)
+     result = val[0] + [val[1]]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 20)
+  def _reduce_5(val, _values, result)
+     result = val[0]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 21)
+  def _reduce_6(val, _values, result)
+     result = val[0]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 22)
+  def _reduce_7(val, _values, result)
+     result = nil
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 27)
+  def _reduce_8(val, _values, result)
+     result = { type: :schema, header: val[0], content: val[3] }
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 29)
+  def _reduce_9(val, _values, result)
+     result = { type: :schema, header: val[0], content: val[3] }
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 34)
+  def _reduce_10(val, _values, result)
+     result = { name: val[0], size: val[2].to_i, fields: val[5] }
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 36)
+  def _reduce_11(val, _values, result)
+     result = { name: val[0], size: nil, fields: val[2] }
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 40)
+  def _reduce_12(val, _values, result)
+     result = [val[0]]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 41)
+  def _reduce_13(val, _values, result)
+     result = val[0] + [val[2]]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 45)
+  def _reduce_14(val, _values, result)
+     result = val[0]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 49)
+  def _reduce_15(val, _values, result)
+     result = { type: :field, name: val[0], value: val[2] }
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 50)
+  def _reduce_16(val, _values, result)
+     result = { type: :data_row, values: val[0] }
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 54)
+  def _reduce_17(val, _values, result)
+     result = val[0]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 58)
+  def _reduce_18(val, _values, result)
+     result = [val[0]]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 59)
+  def _reduce_19(val, _values, result)
+     result = val[0] + [val[2]]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 63)
+  def _reduce_20(val, _values, result)
+     result = val[0]
+    result
+  end
+.,.,
+
+module_eval(<<'.,.,', 'parser.y', 64)
+  def _reduce_21(val, _values, result)
+     result = val[0]
+    result
+  end
+.,.,
+
+def _reduce_none(val, _values, result)
+  val[0]
+end
+
+  end   # class Parser
+end   # module Rtoon

data/lib/rtoon/parser.y

@@ -0,0 +1,154 @@
+# TOON Grammar - Tabular, schema-based format
+
+class Rtoon::Parser
+
+token IDENTIFIER NUMBER
+token LBRACKET RBRACKET LBRACE RBRACE COLON COMMA
+token INDENT DEDENT NEWLINE
+
+rule
+  document
+    : statements { result = process_statements(val[0]) }
+    | /* empty */ { result = {} }
+    ;
+
+  statements
+    : statement { result = [val[0]] }
+    | statements statement { result = val[0] + [val[1]] }
+    ;
+
+  statement
+    : schema_block { result = val[0] }
+    | field_assignment { result = val[0] }
+    | NEWLINE { result = nil }
+    ;
+
+  schema_block
+    : schema_header NEWLINE INDENT block_content DEDENT
+      { result = { type: :schema, header: val[0], content: val[3] } }
+    | schema_header NEWLINE INDENT block_content
+      { result = { type: :schema, header: val[0], content: val[3] } }
+    ;
+
+  schema_header
+    : IDENTIFIER LBRACKET NUMBER RBRACKET LBRACE field_list RBRACE COLON
+      { result = { name: val[0], size: val[2].to_i, fields: val[5] } }
+    | IDENTIFIER LBRACE field_list RBRACE COLON
+      { result = { name: val[0], size: nil, fields: val[2] } }
+    ;
+
+  field_list
+    : IDENTIFIER { result = [val[0]] }
+    | field_list COMMA IDENTIFIER { result = val[0] + [val[2]] }
+    ;
+
+  block_content
+    : statements { result = val[0] }
+    ;
+
+  field_assignment
+    : IDENTIFIER COLON value { result = { type: :field, name: val[0], value: val[2] } }
+    | data_row { result = { type: :data_row, values: val[0] } }
+    ;
+
+  data_row
+    : value_list { result = val[0] }
+    ;
+
+  value_list
+    : value { result = [val[0]] }
+    | value_list COMMA value { result = val[0] + [val[2]] }
+    ;
+
+  value
+    : IDENTIFIER { result = val[0] }
+    | NUMBER { result = val[0] }
+    ;
+
+end
+
+---- header
+require_relative 'lexer'
+
+---- inner
+
+def parse(str)
+  @lexer = Lexer.new(str)
+  do_parse
+end
+
+def next_token
+  @lexer.next_token
+end
+
+def on_error(token_id, value, value_stack)
+  line = @lexer.line_number
+  raise ParseError, "Parse error at line #{line}: unexpected token #{value.inspect}"
+end
+
+def process_statements(statements)
+  result = {}
+  current_schema = nil
+  data_rows = []
+
+  statements.compact.each do |stmt|
+    case stmt[:type]
+    when :schema
+      # First, finalize any pending schema with data rows
+      if current_schema && !data_rows.empty?
+        result[current_schema[:name]] = build_array_from_rows(current_schema, data_rows)
+        data_rows = []
+      end
+
+      current_schema = stmt[:header]
+
+      # Process nested content
+      nested_result = process_statements(stmt[:content])
+
+      # Check if there are data rows in the nested content
+      nested_data_rows = stmt[:content].select { |s| s && s[:type] == :data_row }
+
+      if !nested_data_rows.empty?
+        # This schema has data rows
+        result[current_schema[:name]] = build_array_from_rows(current_schema, nested_data_rows)
+        current_schema = nil
+      elsif !nested_result.empty?
+        # This schema has nested schemas
+        if current_schema[:size]
+          result[current_schema[:name]] = [nested_result]
+        else
+          result[current_schema[:name]] = nested_result
+        end
+        current_schema = nil
+      end
+
+    when :field
+      result[stmt[:name]] = stmt[:value]
+
+    when :data_row
+      data_rows << stmt[:values]
+    end
+  end
+
+  # Finalize any remaining schema
+  if current_schema && !data_rows.empty?
+    result[current_schema[:name]] = build_array_from_rows(current_schema, data_rows)
+  end
+
+  result
+end
+
+def build_array_from_rows(schema, data_row_statements)
+  fields = schema[:fields]
+
+  data_row_statements.map do |row_stmt|
+    values = row_stmt[:values]
+    obj = {}
+    fields.each_with_index do |field, idx|
+      obj[field] = values[idx] if idx < values.length
+    end
+    obj
+  end
+end
+
+class ParseError < StandardError; end

data/lib/rtoon.rb

@@ -0,0 +1,19 @@
+require_relative 'rtoon/parser.tab'
+require_relative 'rtoon/encoder'
+
+module Rtoon
+  VERSION = "0.1.0"
+
+  def self.parse(string)
+    parser = Parser.new
+    parser.parse(string)
+  end
+
+  def self.decode(string)
+    parse(string)
+  end
+
+  def self.encode(hash, indent_level = 0)
+    Encoder.encode(hash, indent: indent_level)
+  end
+end

metadata

@@ -0,0 +1,91 @@
+--- !ruby/object:Gem::Specification
+name: rtoon
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Antonio Chavez
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: racc
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.7'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.7'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+description: A Ruby library for parsing TOON, a tabular, schema-based data format
+  with indentation-based structure
+email:
+- antonio@zoobean.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- LICENSE.txt
+- README.md
+- TOON_SPEC.md
+- lib/rtoon.rb
+- lib/rtoon/encoder.rb
+- lib/rtoon/lexer.rb
+- lib/rtoon/parser.tab.rb
+- lib/rtoon/parser.y
+homepage: https://github.com/zoobean/rtoon
+licenses:
+- MIT
+metadata: {}
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.7.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.6.7
+specification_version: 4
+summary: Parser for Token Object Oriented Notation (TOON)
+test_files: []