toon-parser 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 146ba580e5b40ed0c57aca735d092c02660125aad2e0ca06643da9336e18d49d
+  data.tar.gz: 7e86260004907ad3d34769751964928a88e7712a478e6d0dc628affc9e883ea8
+SHA512:
+  metadata.gz: e1bacf72b9ba9bcd27934039294b5a1fe16975a75b28e0b70cb7eb7ce8787292fa55247f5a3b2ab9a178d77ca7ab2d7c62cd68f7b148b6567591a38b5a9b5484
+  data.tar.gz: ea087bc89093ced0c234f1a76acf962eb1ed934a31a7e7882b053f9dc3eface801692f24a74dc95e8574faba4f0d8c74a73c57346c3d8b1d0c50f19a188b4dcf

data/.rspec

@@ -0,0 +1,4 @@
+--require spec_helper
+--format documentation
+--color
+

data/CHANGELOG.md

@@ -0,0 +1,39 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2024-01-XX
+
+### Added
+- Initial release of ToonParser gem
+- Core parsing functionality for TOON format
+- Serialization of Ruby objects to TOON format
+- File operations (`parse_file`, `serialize_to_file`)
+- Rails integration with Railtie
+- Controller helpers for TOON responses
+- Automatic MIME type registration (`application/toon`)
+- Parameter parser for TOON request bodies
+- API client helpers for TOON-formatted HTTP requests
+- Support for nested objects
+- Support for arrays of objects with schemas
+- Support for simple arrays
+- Type inference for numbers, booleans, and null values
+- Comprehensive test suite
+- Documentation and examples
+
+### Features
+- Parse TOON strings to Ruby objects (Hash/Array)
+- Serialize Ruby objects to TOON format
+- Rails controller integration with `render toon: data`
+- Automatic request body parsing for TOON format
+- HTTP client for TOON APIs
+- Round-trip conversion support
+
+[Unreleased]: https://github.com/afshmini/toon-parser/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/afshmini/toon-parser/releases/tag/v0.1.0
+

data/CONTRIBUTING.md

@@ -0,0 +1,77 @@
+# Contributing to ToonParser
+
+Thank you for your interest in contributing to ToonParser! This document provides guidelines and instructions for contributing.
+
+## How to Contribute
+
+### Reporting Bugs
+
+If you find a bug, please open an issue on GitHub with:
+
+- A clear, descriptive title
+- Steps to reproduce the issue
+- Expected behavior
+- Actual behavior
+- Ruby version and gem version
+- Any relevant code examples
+
+### Suggesting Features
+
+Feature suggestions are welcome! Please open an issue with:
+
+- A clear description of the feature
+- Use cases and examples
+- Any potential implementation ideas
+
+### Pull Requests
+
+1. **Fork the repository** and create your branch from `main`
+2. **Make your changes** following the coding standards
+3. **Add tests** for new functionality
+4. **Ensure all tests pass** (`bundle exec rspec`)
+5. **Update documentation** if needed
+6. **Submit a pull request** with a clear description
+
+## Development Setup
+
+```bash
+# Clone your fork
+git clone https://github.com/YOUR_USERNAME/toon-parser.git
+cd toon-parser
+
+# Install dependencies
+bundle install
+
+# Run tests
+bundle exec rspec
+
+# Run linter
+bundle exec rubocop
+```
+
+## Coding Standards
+
+- Follow Ruby style guidelines
+- Write tests for new features
+- Keep code simple and readable
+- Add documentation for public methods
+- Use meaningful variable and method names
+
+## Testing
+
+- Write tests for all new features
+- Ensure existing tests still pass
+- Aim for good test coverage
+- Test edge cases
+
+## Documentation
+
+- Update README.md for user-facing changes
+- Add code comments for complex logic
+- Update examples if API changes
+- Keep CHANGELOG.md updated
+
+## Questions?
+
+Feel free to open an issue for any questions about contributing!
+

data/LICENSE.txt

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2024 afshmini
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

data/README.md

@@ -0,0 +1,437 @@
+# ToonParser
+
+[![Ruby Version](https://img.shields.io/badge/ruby-%3E%3D%202.7.0-red.svg)](https://www.ruby-lang.org/)
+[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE.txt)
+[![GitHub](https://img.shields.io/github/stars/afshmini/toon-parser?style=social)](https://github.com/afshmini/toon-parser)
+
+A Ruby gem for parsing and serializing **TOON** (Token-Oriented Object Notation) format. TOON is a compact, human-readable data format designed to reduce token usage in LLM applications by **30-60%** compared to JSON.
+
+## Table of Contents
+
+- [Features](#features)
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Usage](#usage)
+  - [Basic Parsing](#basic-parsing)
+  - [Basic Serialization](#basic-serialization)
+  - [File Operations](#file-operations)
+  - [Rails Controller Integration](#rails-controller-integration)
+  - [API Integration](#api-integration)
+  - [Advanced Usage](#advanced-usage)
+- [TOON Format Overview](#toon-format-overview)
+- [Examples](#examples)
+- [Development](#development)
+- [Contributing](#contributing)
+- [License](#license)
+
+## Features
+
+✨ **Key Features:**
+
+- 🚀 **Fast parsing and serialization** of TOON format
+- 🎯 **Rails integration** - Use TOON in controllers instead of JSON
+- 📦 **File operations** - Read and write TOON files
+- 🌐 **API helpers** - Built-in HTTP client for TOON APIs
+- 🔄 **Round-trip conversion** - Parse and serialize without data loss
+- 🎨 **Type inference** - Automatic detection of numbers, booleans, and null
+- 📝 **Human-readable** - Compact yet readable format
+- 🔌 **Optional Rails dependency** - Works standalone or with Rails
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'toon-parser'
+```
+
+And then execute:
+
+```bash
+$ bundle install
+```
+
+Or install it yourself as:
+
+```bash
+$ gem install toon-parser
+```
+
+## Quick Start
+
+```ruby
+require 'toon-parser'
+
+# Parse TOON string
+toon = "users[2]{id,name}:\n  1,Alice\n  2,Bob"
+data = ToonParser.parse(toon)
+# => {"users" => [{"id" => 1, "name" => "Alice"}, {"id" => 2, "name" => "Bob"}]}
+
+# Serialize to TOON
+data = { "users" => [{ "id" => 1, "name" => "Alice" }] }
+toon = ToonParser.serialize(data)
+# => "users[1]{id,name}:\n  1,Alice"
+```
+
+## Usage
+
+### Basic Parsing
+
+Parse a TOON string into Ruby objects:
+
+```ruby
+require 'toon-parser'
+
+toon = <<~TOON
+  users[2]{id,name}:
+    1,Alice
+    2,Bob
+TOON
+
+data = ToonParser.parse(toon)
+# => {"users" => [{"id" => 1, "name" => "Alice"}, {"id" => 2, "name" => "Bob"}]}
+```
+
+### Basic Serialization
+
+Convert Ruby objects to TOON format:
+
+```ruby
+data = {
+  "users" => [
+    { "id" => 1, "name" => "Alice" },
+    { "id" => 2, "name" => "Bob" }
+  ]
+}
+
+toon = ToonParser.serialize(data)
+# => "users[2]{id,name}:\n  1,Alice\n  2,Bob"
+```
+
+### File Operations
+
+Parse a TOON file:
+
+```ruby
+data = ToonParser.parse_file("data.toon")
+```
+
+Serialize to a TOON file:
+
+```ruby
+ToonParser.serialize_to_file(data, "output.toon")
+```
+
+### Rails Controller Integration
+
+Use TOON format in your Rails controllers instead of JSON. The gem automatically integrates with Rails when available.
+
+#### Basic Controller Usage
+
+```ruby
+# app/controllers/api/users_controller.rb
+class Api::UsersController < ApplicationController
+  before_action :set_toon_format
+
+  def index
+    @users = User.all
+    render toon: { 
+      users: @users.map { |u| { id: u.id, name: u.name, email: u.email } } 
+    }
+  end
+
+  def show
+    @user = User.find(params[:id])
+    render toon: { 
+      user: { id: @user.id, name: @user.name, email: @user.email } 
+    }
+  end
+
+  def create
+    # TOON request body is automatically parsed if Content-Type is application/toon
+    user_data = parse_toon_request
+    @user = User.create(user_data)
+    
+    if @user.persisted?
+      render toon: { user: @user }, status: :created
+    else
+      render toon: { errors: @user.errors.full_messages }, status: :unprocessable_entity
+    end
+  end
+
+  private
+
+  def set_toon_format
+    request.format = :toon if request.format == :html
+  end
+end
+```
+
+#### Setting Default Format
+
+Set TOON as the default format for API controllers:
+
+```ruby
+# app/controllers/api/base_controller.rb
+class Api::BaseController < ApplicationController
+  before_action :set_toon_format
+
+  private
+
+  def set_toon_format
+    request.format = :toon if request.format == :html
+  end
+end
+```
+
+#### Responding to Multiple Formats
+
+Support both TOON and JSON:
+
+```ruby
+class Api::UsersController < ApplicationController
+  def index
+    @users = User.all
+    data = { users: @users.map { |u| { id: u.id, name: u.name } } }
+    
+    respond_to do |format|
+      format.toon { render toon: data }
+      format.json { render json: data }
+    end
+  end
+end
+```
+
+#### Parsing TOON Request Bodies
+
+The gem automatically registers a parameter parser for TOON request bodies. When a request has `Content-Type: application/toon`, Rails will automatically parse the body. You can also manually parse:
+
+```ruby
+class Api::UsersController < ApplicationController
+  def create
+    # Option 1: Automatic parsing (if Content-Type is application/toon)
+    # The body is automatically parsed into params
+    
+    # Option 2: Manual parsing
+    user_data = parse_toon_request
+    @user = User.create(user_data)
+    render toon: { user: @user }
+  end
+end
+```
+
+**Note:** The automatic parameter parser only works if the request has `Content-Type: application/toon` header. For manual parsing, use the `parse_toon_request` helper method.
+
+#### Routes Configuration
+
+Configure routes to accept TOON format:
+
+```ruby
+# config/routes.rb
+Rails.application.routes.draw do
+  namespace :api do
+    resources :users, defaults: { format: :toon }
+  end
+end
+```
+
+#### Available Controller Helpers
+
+- `render_toon(data, options = {})` - Render TOON response
+- `parse_toon_request` - Parse TOON request body
+- `set_toon_format` - Set TOON as default format
+- `respond_to_toon(data, options = {})` - Support multiple formats
+
+### API Integration
+
+Use the API helper for TOON-formatted API requests and responses:
+
+```ruby
+# Create an API client
+api = ToonParser.api("https://api.example.com", headers: { "Authorization" => "Bearer token" })
+
+# GET request
+users = api.get("/users")
+
+# POST request with TOON body
+new_user = api.post("/users", data: { "name" => "Alice", "email" => "alice@example.com" })
+
+# PUT request
+updated = api.put("/users/1", data: { "name" => "Alice Updated" })
+
+# PATCH request
+patched = api.patch("/users/1", data: { "name" => "Alice Patched" })
+
+# DELETE request
+api.delete("/users/1")
+```
+
+### Advanced Usage
+
+#### Nested Objects
+
+```ruby
+toon = <<~TOON
+  user:
+    id: 1
+    name: Alice
+    profile:
+      age: 30
+      city: New York
+TOON
+
+data = ToonParser.parse(toon)
+# => {"user" => {"id" => 1, "name" => "Alice", "profile" => {"age" => 30, "city" => "New York"}}}
+```
+
+#### Arrays of Primitives
+
+```ruby
+toon = <<~TOON
+  numbers[3]:
+    1
+    2
+    3
+TOON
+
+data = ToonParser.parse(toon)
+# => {"numbers" => [1, 2, 3]}
+```
+
+#### Boolean and Null Values
+
+```ruby
+toon = <<~TOON
+  data:
+    active: true
+    deleted: false
+    value: null
+TOON
+
+data = ToonParser.parse(toon)
+# => {"data" => {"active" => true, "deleted" => false, "value" => nil}}
+```
+
+#### Floating Point Numbers
+
+```ruby
+toon = <<~TOON
+  prices[2]:
+    19.99
+    29.50
+TOON
+
+data = ToonParser.parse(toon)
+# => {"prices" => [19.99, 29.50]}
+```
+
+## TOON Format Overview
+
+TOON uses a compact syntax that eliminates unnecessary brackets, quotes, and whitespace:
+
+**JSON:**
+```json
+{
+  "users": [
+    { "id": 1, "name": "Alice" },
+    { "id": 2, "name": "Bob" }
+  ]
+}
+```
+
+**TOON:**
+```
+users[2]{id,name}:
+  1,Alice
+  2,Bob
+```
+
+### Key Features
+
+- **Array notation**: `[size]` indicates array length
+- **Object schema**: `{field1,field2}` defines object structure
+- **Tabular data**: Rows are comma-separated values
+- **Type inference**: Numbers, booleans, and null are automatically detected
+- **Indentation**: Used for nesting and structure
+- **Compact**: 30-60% smaller than JSON for typical data structures
+
+## Examples
+
+See the [examples directory](examples/) for more usage examples:
+
+- [Rails Controller Example](examples/rails_controller_example.rb) - Complete Rails controller implementation
+
+## Development
+
+After checking out the repo, run:
+
+```bash
+$ bundle install
+```
+
+To run the test suite:
+
+```bash
+$ bundle exec rspec
+```
+
+Or run tests directly:
+
+```bash
+$ rspec spec/
+```
+
+To install this gem onto your local machine, run:
+
+```bash
+$ bundle exec rake install
+```
+
+To release a new version:
+
+1. Update the version number in `lib/toon-parser/version.rb`
+2. Run `bundle exec rake release`
+
+This will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add some amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+### Development Setup
+
+```bash
+# Clone the repository
+git clone https://github.com/afshmini/toon-parser.git
+cd toon-parser
+
+# Install dependencies
+bundle install
+
+# Run tests
+bundle exec rspec
+
+# Run linter
+bundle exec rubocop
+```
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](LICENSE.txt).
+
+## Links
+
+- [GitHub Repository](https://github.com/afshmini/toon-parser)
+- [Issue Tracker](https://github.com/afshmini/toon-parser/issues)
+- [RubyGems](https://rubygems.org/gems/toon-parser) (when published)
+
+## Acknowledgments
+
+- Inspired by the [TOON format specification](https://github.com/toon-format/spec)
+- Designed to reduce token usage in LLM applications

data/Rakefile

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec
+

data/examples/rails_controller_example.rb

@@ -0,0 +1,89 @@
+# Example Rails Controller using TOON format
+# Place this in app/controllers/api/users_controller.rb
+
+class Api::UsersController < ApplicationController
+  # Set TOON as default format for this controller
+  before_action :set_toon_format
+
+  def index
+    @users = User.all
+    # Render as TOON format
+    render toon: {
+      users: @users.map do |user|
+        {
+          id: user.id,
+          name: user.name,
+          email: user.email,
+          created_at: user.created_at
+        }
+      end
+    }
+  end
+
+  def show
+    @user = User.find(params[:id])
+    render toon: {
+      user: {
+        id: @user.id,
+        name: @user.name,
+        email: @user.email,
+        profile: {
+          bio: @user.bio,
+          avatar: @user.avatar_url
+        }
+      }
+    }
+  end
+
+  def create
+    # Parse TOON request body
+    user_data = parse_toon_request
+    
+    @user = User.new(user_data)
+    if @user.save
+      render toon: { user: @user }, status: :created
+    else
+      render toon: { errors: @user.errors.full_messages }, status: :unprocessable_entity
+    end
+  end
+
+  def update
+    @user = User.find(params[:id])
+    user_data = parse_toon_request
+    
+    if @user.update(user_data)
+      render toon: { user: @user }
+    else
+      render toon: { errors: @user.errors.full_messages }, status: :unprocessable_entity
+    end
+  end
+
+  def destroy
+    @user = User.find(params[:id])
+    @user.destroy
+    render toon: { message: "User deleted successfully" }, status: :ok
+  end
+
+  private
+
+  def set_toon_format
+    # Set TOON as default format if not specified
+    request.format = :toon if request.format == :html
+  end
+end
+
+# Example: Support both TOON and JSON
+class Api::ProductsController < ApplicationController
+  def index
+    @products = Product.all
+    data = {
+      products: @products.map { |p| { id: p.id, name: p.name, price: p.price } }
+    }
+
+    respond_to do |format|
+      format.toon { render toon: data }
+      format.json { render json: data }
+    end
+  end
+end
+