unibuf 0.1.1 → 0.1.2

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,132 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official email address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[INSERT CONTACT METHOD].
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

data/README.adoc

@@ -1,4 +1,4 @@
-= Unibuf: Universal Protocol Buffer & FlatBuffer Parser
+= Unibuf: Universal Buffer Format Parser
 
 image:https://img.shields.io/gem/v/unibuf.svg[Gem Version,link=https://rubygems.org/gems/unibuf]
 image:https://img.shields.io/github/license/lutaml/unibuf.svg[License,link=https://github.com/lutaml/unibuf/blob/main/LICENSE]
@@ -6,21 +6,43 @@ image:https://github.com/lutaml/unibuf/actions/workflows/rake.yml/badge.svg[Buil
 
 == Purpose
 
-Unibuf is a pure Ruby gem for parsing and manipulating Protocol Buffers in both text and binary formats with schema-driven validation.
+Unibuf is a pure Ruby gem for parsing and manipulating multiple serialization
+formats including Protocol Buffers, FlatBuffers, and Cap'n Proto.
 
-It provides a fully object-oriented, specification-compliant parser with rich domain models, comprehensive schema validation, wire format decoding, and complete round-trip serialization support.
+It provides fully object-oriented, specification-compliant parsers with rich
+domain models, comprehensive schema validation, binary format encoding/decoding,
+and complete round-trip serialization support.
 
 Key features:
 
-* Parse Protocol Buffers text format (`.txtpb`, `.textproto`)
-* Parse Protocol Buffers binary format (`.binpb`) with schema
-* Parse Proto3 schemas (`.proto`) for validation
-* Schema-driven validation and deserialization
-* Wire format decoding (varint, zigzag, all wire types)
-* Round-trip serialization with 100% accuracy
-* Rich domain models with 45+ behavioral classes
-* Complete CLI toolkit for text and binary formats
-* Specification-compliant implementation
+* Protocol Buffers
+** Parse text format (`.txtpb`, `.textproto`)
+** Parse binary format (`.binpb`) with schema
+** Serialize to binary format (`.binpb`)
+** Parse Proto3 schemas (`.proto`)
+** Wire format encoding/decoding (varint, zigzag, all wire types)
+
+* FlatBuffers
+** Parse schemas (`.fbs`)
+** Parse binary format (`.fb`)
+** Serialize to binary format (`.fb`)
+
+* Cap'n Proto
+** Parse schemas (`.capnp`)
+** Parse binary format with segment management
+** Serialize to binary format with pointer encoding
+** Support for structs, enums, interfaces (RPC)
+** Generic types (List<T>)
+** Unions and annotations
+
+* Serialization and validation
+** Complete round-trip serialization for all formats
+** Schema-driven validation and deserialization
+
+* Developer usage
+** Rich domain models with 60+ behavioral classes
+** Complete CLI toolkit for all formats
+** Pure Ruby - no C/C++ dependencies
 
 == Installation
 
@@ -47,6 +69,9 @@ gem install unibuf
 
 == Features
 
+* <<protocol-buffers,Protocol Buffers support>>
+* <<flatbuffers,FlatBuffers support>>
+* <<capnproto,Cap'n Proto support>>
 * <<schema-required-design,Schema-required design>>
 * <<parsing-textproto,Parsing text format>>
 * <<parsing-binary,Parsing binary format>>
@@ -56,32 +81,182 @@ gem install unibuf
 * <<rich-domain-models,Rich domain models>>
 * <<cli-tools,Command-line tools>>
 
-[[schema-required-design]]
-== Schema-required design
+[[protocol-buffers]]
+== Protocol Buffers
 
 === General
 
-Unibuf follows Protocol Buffers' schema-driven architecture. The schema (`.proto` file) defines the message structure and is REQUIRED for binary parsing and recommended for text parsing.
+Full support for Protocol Buffers (protobuf) including text format parsing,
+binary format parsing/serialization, and Proto3 schema parsing.
 
-This design ensures type safety and enables proper deserialization of binary formats.
+See link:docs/PROTOBUF.adoc[PROTOBUF.adoc] for detailed documentation.
 
-=== Why schema is required
 
-The schema defines:
-- Message types and their fields
-- Field types and numbers
-- Field wire types for binary encoding
-- Repeated and optional fields
-- Nested message structures
+=== Parsing Protocol Buffers text format
+
+[source,ruby]
+----
+require "unibuf"
+
+# Load schema (recommended for validation)
+schema = Unibuf.parse_schema("schema.proto")  # <1>
+
+# Parse text format file
+message = Unibuf.parse_textproto_file("data.txtpb")  # <2>
+
+# Validate against schema
+validator = Unibuf::Validators::SchemaValidator.new(schema)  # <3>
+validator.validate!(message, "MessageType")  # <4>
+----
+<1> Load Proto3 schema from .proto file
+<2> Parse Protocol Buffers text format
+<3> Create validator with schema
+<4> Validate message against schema
+
+=== Parsing Protocol Buffers binary format
+
+[source,ruby]
+----
+require "unibuf"
+
+# 1. Load schema (REQUIRED for binary)
+schema = Unibuf.parse_schema("schema.proto")  # <1>
+
+# 2. Parse binary Protocol Buffer file
+message = Unibuf.parse_binary_file("data.binpb", schema: schema)  # <2>
+
+# 3. Access fields normally
+puts message.find_field("name").value  # <3>
+----
+<1> Schema is mandatory for binary parsing
+<2> Parse binary file with schema
+<3> Access fields like text format
+
+[[flatbuffers]]
+== FlatBuffers
+
+=== General
+
+Complete support for Google FlatBuffers including schema parsing (`.fbs` files)
+and binary format parsing/serialization.
+
+See link:docs/FLATBUFFERS.adoc[FLATBUFFERS.adoc] for detailed documentation.
+
+=== Parsing FlatBuffers schema
+
+[source,ruby]
+----
+require "unibuf"
+
+# Parse FlatBuffers schema
+schema = Unibuf.parse_flatbuffers_schema("schema.fbs")  # <1>
+
+# Access schema structure
+table = schema.find_table("Monster")  # <2>
+table.fields.each { |f| puts "#{f.name}: #{f.type}" }  # <3>
+----
+<1> Parse `.fbs` schema file
+<2> Find table definition
+<3> Iterate through fields
+
+=== Parsing FlatBuffers binary format
+
+[source,ruby]
+----
+# Parse binary FlatBuffer
+data = Unibuf.parse_flatbuffers_binary(binary_data, schema: schema)  # <1>
+
+# Access data
+puts data["name"]  # <2>
+puts data["hp"]  # <3>
+----
+<1> Parse binary with schema
+<2> Access string field
+<3> Access numeric field
+
+
+[[capnproto]]
+== Cap'n Proto
+
+=== General
+
+Complete support for Cap'n Proto including schema parsing (`.capnp` files) and
+binary format parsing/serialization with segment management and pointer
+encoding.
+
+See link:docs/CAPNPROTO.adoc[CAPNPROTO.adoc] for detailed documentation.
+
+=== Parsing Cap'n Proto schema
+
+[source,ruby]
+----
+require "unibuf"
+
+# Parse Cap'n Proto schema
+schema = Unibuf.parse_capnproto_schema("addressbook.capnp")  # <1>
+
+# Access schema structure
+person = schema.find_struct("Person")  # <2>
+person.fields.each { |f| puts "#{f.name} @#{f.ordinal} :#{f.type}" }  # <3>
+
+# Access interfaces (RPC)
+calc = schema.find_interface("Calculator")  # <4>
+calc.methods.each { |m| puts "#{m.name} @#{m.ordinal}" }  # <5>
+----
+<1> Parse `.capnp` schema file
+<2> Find struct definition
+<3> Iterate through fields with ordinals
+<4> Find interface definition (RPC)
+<5> List RPC methods
+
+=== Parsing Cap'n Proto binary format
+
+[source,ruby]
+----
+# Parse binary Cap'n Proto data
+parser = Unibuf::Parsers::Capnproto::BinaryParser.new(schema)  # <1>
+data = parser.parse(binary_data, root_type: "Person")  # <2>
+
+# Access data
+puts data[:name]  # <3>
+puts data[:email]  # <4>
+----
+<1> Create parser with schema
+<2> Parse binary with root type
+<3> Access text field
+<4> Access another field
+
+=== Serializing Cap'n Proto binary format
+
+[source,ruby]
+----
+# Serialize to binary
+serializer = Unibuf::Serializers::Capnproto::BinarySerializer.new(schema)  # <1>
+binary = serializer.serialize(
+  { id: 1, name: "Alice", email: "alice@example.com" },  # <2>
+  root_type: "Person"  # <3>
+)
+
+# Write to file
+File.binwrite("output.capnp.bin", binary)  # <4>
+----
+<1> Create serializer with schema
+<2> Provide data as hash
+<3> Specify root struct type
+<4> Write binary output
+
 
-Binary Protocol Buffers cannot be parsed without a schema because the binary format only stores field numbers, not field names or types.
 
 [[parsing-textproto]]
-== Parsing Protocol Buffers text format
+== Protocol Buffers text format
 
 === General
 
-Parse human-readable Protocol Buffer text format files following the https://protobuf.dev/reference/protobuf/textformat-spec/[official specification].
+Parse human-readable Protocol Buffer text format files following the
+https://protobuf.dev/reference/protobuf/textformat-spec/[official specification].
+
+See link:docs/TXTPROTO.adoc[TXTPROTO.adoc] for detailed documentation.
+
 
 === Parsing text format
 
@@ -109,9 +284,11 @@ validator.validate!(message, "MessageType")  # <4>
 
 === General
 
-Parse binary Protocol Buffer data using wire format decoding with schema-driven deserialization.
+Parse binary Protocol Buffer data using wire format decoding with schema-driven
+deserialization.
 
-The schema is REQUIRED for binary parsing because binary format only stores field numbers, not names or types.
+The schema is REQUIRED for binary parsing because binary format only stores
+field numbers, not names or types.
 
 === Parsing binary format
 
@@ -160,46 +337,14 @@ Variable-length data: string, bytes, embedded messages, packed repeated fields
 32-bit (Type 5)::
 Fixed 4-byte values: fixed32, sfixed32, float
 
-[[schema-validation]]
-== Schema-based validation
-
-=== General
-
-Validate Protocol Buffer messages (text or binary) against their Proto3 schemas.
-
-=== Validating with schema
-
-[source,ruby]
-----
-# Load schema
-schema = Unibuf.parse_schema("schema.proto")  # <1>
-
-# Parse message (text or binary)
-message = Unibuf.parse_binary_file("data.binpb", schema: schema)  # <2>
-
-# Validate
-validator = Unibuf::Validators::SchemaValidator.new(schema)  # <3>
-errors = validator.validate(message, "MessageType")  # <4>
-
-if errors.empty?
-  puts "✓ Valid!"  # <5>
-else
-  errors.each { |e| puts "  - #{e}" }  # <6>
-end
-----
-<1> Parse the Proto3 schema
-<2> Parse binary Protocol Buffer
-<3> Create validator with schema
-<4> Validate message
-<5> Validation passed
-<6> Show errors if any
 
 [[wire-format]]
-== Wire format support
+== Protocol Buffers wire format
 
 === General
 
-Unibuf implements complete Protocol Buffers wire format decoding according to the official specification.
+Unibuf implements complete Protocol Buffers wire format decoding according to
+the official specification.
 
 === Wire format features
 
@@ -236,12 +381,78 @@ message.field_names  # => ["name", "id", "enabled"]
 message.find_field("id").value  # => Properly decoded integer
 ----
 
+
+
+[[schema-required-design]]
+== Schema-required design
+
+=== General
+
+Unibuf follows Protocol Buffers' and FlatBuffers' schema-driven architecture.
+The schema (`.proto` or `.fbs` file) defines the message structure and is
+REQUIRED for binary parsing and serialization.
+
+This design ensures type safety and enables proper deserialization of binary
+formats.
+
+=== Why schema is required
+
+The schema defines:
+
+* Message/struct types and their fields
+* Field types, numbers, and ordinals
+* Field wire types for binary encoding
+* Repeated and optional fields
+* Nested message/struct structures
+
+Binary Protocol Buffers, FlatBuffers, and Cap'n Proto cannot be parsed without a
+schema because the binary formats only store field identifiers, not field names
+or complete type information.
+
+
+[[schema-validation]]
+== Schema-based validation
+
+=== General
+
+Validate Protocol Buffer messages (text or binary) against their Proto3 schemas.
+
+=== Validating with schema
+
+[source,ruby]
+----
+# Load schema
+schema = Unibuf.parse_schema("schema.proto")  # <1>
+
+# Parse message (text or binary)
+message = Unibuf.parse_binary_file("data.binpb", schema: schema)  # <2>
+
+# Validate
+validator = Unibuf::Validators::SchemaValidator.new(schema)  # <3>
+errors = validator.validate(message, "MessageType")  # <4>
+
+if errors.empty?
+  puts "✓ Valid!"  # <5>
+else
+  errors.each { |e| puts "  - #{e}" }  # <6>
+end
+----
+<1> Parse the Proto3 schema
+<2> Parse binary Protocol Buffer
+<3> Create validator with schema
+<4> Validate message
+<5> Validation passed
+<6> Show errors if any
+
+
+
 [[round-trip-serialization]]
 == Round-trip serialization
 
 === General
 
-Unibuf supports complete round-trip serialization for text format, allowing you to parse, modify, and serialize back while preserving semantic equivalence.
+Unibuf supports complete round-trip serialization for text format, allowing you
+to parse, modify, and serialize back while preserving semantic equivalence.
 
 === Serializing to textproto format
 
@@ -272,7 +483,7 @@ puts message == reparsed  # => true <5>
 
 Unibuf provides rich domain models with comprehensive behavior.
 
-Over 45 classes provide extensive functionality following object-oriented principles.
+Over 60 classes provide extensive functionality following object-oriented principles.
 
 === Message model
 
@@ -386,8 +597,30 @@ Unibuf
 │   │   ├── Grammar        Proto3 grammar
 │   │   ├── Processor      Schema builder
 │   │   └── Parser         Schema API
-│   └── Binary             Binary format parser
-│       └── WireFormatParser   Wire format decoder
+│   ├── Binary             Binary Protocol Buffers
+│   │   └── WireFormatParser   Wire format decoder
+│   ├── Flatbuffers        FlatBuffers parser
+│   │   ├── Grammar        FBS grammar
+│   │   ├── Processor      Schema builder
+│   │   └── BinaryParser   Binary format
+│   └── Capnproto          Cap'n Proto parser
+│       ├── Grammar        Cap'n Proto grammar
+│       ├── Processor      Schema builder
+│       ├── SegmentReader  Segment management
+│       ├── PointerDecoder Pointer decoding
+│       ├── StructReader   Struct reading
+│       ├── ListReader     List reading
+│       └── BinaryParser   Binary format
+├── Serializers
+│   ├── BinarySerializer   Protocol Buffers binary
+│   ├── Flatbuffers        FlatBuffers binary
+│   │   └── BinarySerializer
+│   └── Capnproto          Cap'n Proto binary
+│       ├── SegmentBuilder Segment allocation
+│       ├── PointerEncoder Pointer encoding
+│       ├── StructWriter   Struct writing
+│       ├── ListWriter     List writing
+│       └── BinarySerializer
 ├── Models
 │   ├── Message            Protocol Buffer message
 │   ├── Field              Message field
@@ -395,6 +628,8 @@ Unibuf
 │   ├── MessageDefinition  Message type definition
 │   ├── FieldDefinition    Field specification
 │   ├── EnumDefinition     Enum type definition
+│   ├── Flatbuffers        FlatBuffers models (6 classes)
+│   ├── Capnproto          Cap'n Proto models (7 classes)
 │   └── Values             Value type hierarchy (5 classes)
 ├── Validators
 │   ├── TypeValidator      Type and range validation
@@ -403,41 +638,6 @@ Unibuf
     └── Commands           parse, validate, convert, schema
 ----
 
-=== Design principles
-
-Object-Oriented::
-45+ rich classes with extensive behavior.
-No anemic data structures.
-
-MECE::
-Mutually exclusive, collectively exhaustive classifications.
-Complete type hierarchies.
-
-Separation of Concerns::
-Clean layer separation: Grammar, Processor, Models, Validators, CLI.
-
-Open/Closed::
-Extensible for new formats without modifying core.
-
-Schema-Driven::
-Schema-required for binary, recommended for text.
-Proper Protocol Buffer architecture.
-
-== Real-World Validation
-
-Curated test suite with diverse Protocol Buffer features:
-
-.Test fixtures
-[example]
-====
-- robotoflex: Multi-axis variable font
-- mavenpro: Static font
-- opensans: Popular font variants
-- playfair: Optical size axis
-- wavefont: Custom axes
-
-Validation: 100% parse success, 100% round-trip accuracy
-====
 
 == Development
 
@@ -457,23 +657,15 @@ bundle exec rubocop -A
 
 == Roadmap
 
-=== Current Version (v1.0.0)
-
-- ✅ Text format parsing
-- ✅ Binary format parsing (wire format decoder)
-- ✅ Proto3 schema parsing
-- ✅ Schema-based validation
-- ✅ Complete CLI toolkit
-- ✅ 277 comprehensive tests
-
 === Future work
 
-==== FlatBuffers
+==== Additional features
 
-- FlatBuffers schema parsing
-- FlatBuffers binary parsing
+- gRPC support (Protocol Buffers RPC)
+- Cap'n Proto RPC implementation
 - Performance optimizations
 - Additional Protocol Buffer features
+- Schema evolution tools
 
 == Contributing