messagepack 1.0.0

data/docs/_references/extensions.adoc

@@ -0,0 +1,198 @@
+---
+title: Extension types reference
+nav_order: 3
+---
+
+== Purpose
+
+Reference for built-in and custom extension types in MessagePack Ruby.
+
+== Concepts
+
+Extension types allow serialization of objects that don't have native MessagePack
+representations. Each extension type has:
+
+* **Type ID** - An integer from -128 to 127
+* **Packer** - Method to convert object to binary
+* **Unpacker** - Method to reconstruct object from binary
+
+== Built-in extension types
+
+=== Timestamp (-1)
+
+Type ID: -1 (reserved by MessagePack specification)
+
+Serializes Time objects with nanosecond precision.
+
+==== Format selection
+
+The implementation automatically selects the most compact format:
+
+* **Timestamp32** (4 bytes) - Seconds only, fits in 32 bits
+* **Timestamp64** (8 bytes) - Seconds + nanoseconds, fits in 64 bits
+* **Timestamp96** (12 bytes) - Full precision
+
+[source,ruby]
+----
+factory.register_type(-1, Time,
+  packer: Messagepack::Time::Packer,
+  unpacker: Messagepack::Time::Unpacker
+)
+----
+
+==== Registration
+
+The default factory includes timestamp registration:
+
+[source,ruby]
+----
+# Already registered in DefaultFactory
+Messagepack::DefaultFactory.register_type(-1, Time,
+  packer: Messagepack::Time::Packer,
+  unpacker: Messagepack::Time::Unpacker
+)
+----
+
+==== Example
+
+[source,ruby]
+----
+# Current time with nanoseconds
+now = Time.now
+binary = factory.pack(now)
+restored = factory.unpack(binary)
+
+# Nanoseconds are preserved
+puts restored.tv_nsec
+----
+
+=== Symbol (0)
+
+Type ID: 0 (common convention, not reserved)
+
+Efficiently serializes Ruby symbols.
+
+[source,ruby]
+----
+factory.register_type(0, Symbol,
+  packer: ->(sym) { sym.to_s },
+  unpacker: ->(str) { str.to_sym }
+)
+----
+
+NOTE: The symbol extension is not registered by default.
+
+==== Example
+
+[source,ruby]
+----
+factory = Messagepack::Factory.new
+factory.register_type(0, Symbol)
+
+data = { status: :active, tags: [:urgent, :important] }
+binary = factory.pack(data)
+result = factory.unpack(binary)
+# => {:status=>:active, :tags=>[:urgent, :important]}
+----
+
+== Custom extension types
+
+=== Type ID ranges
+
+* `-128 to -1` - Reserved for future MessagePack specifications
+* `0` - Convention for symbol extension
+* `1 to 127` - Available for application-specific types
+
+=== Registration patterns
+
+==== Method name (simple)
+
+[source,ruby]
+----
+class MyClass
+  def to_msgpack_ext
+    # Return binary string
+    [value].pack("I")
+  end
+
+  def self.from_msgpack_ext(data)
+    # Parse and return instance
+    new(data.unpack("I").first)
+  end
+end
+
+factory.register_type(0x10, MyClass,
+  packer: :to_msgpack_ext,
+  unpacker: :from_msgpack_ext
+)
+----
+
+==== Proc (flexible)
+
+[source,ruby]
+----
+factory.register_type(0x11, URI,
+  packer: ->(uri) { uri.to_s },
+  unpacker: ->(str) { URI.parse(str) }
+)
+----
+
+==== Recursive (nested structures)
+
+[source,ruby]
+----
+factory.register_type(0x12, TreeNode,
+  packer: ->(obj, packer) {
+    packer.write({ value: obj.value, left: obj.left, right: obj.right })
+  },
+  unpacker: ->(unpacker) {
+    data = unpacker.read
+    TreeNode.new(data[:value], data[:left], data[:right])
+  },
+  recursive: true
+)
+----
+
+== Extension value class
+
+The `Messagepack::ExtensionValue` class represents raw extension types.
+
+=== Accessing extension data
+
+[source,ruby]
+----
+# Register extension that produces ExtensionValue
+factory.register_type(0x20, MyClass, ...)
+
+# When unpacking, access raw extension data
+obj = factory.unpack(binary)
+if obj.is_a?(Messagepack::ExtensionValue)
+  puts "Type: #{obj.type}"
+  puts "Data: #{obj.data.inspect}"
+end
+----
+
+=== Creating extension values
+
+[source,ruby]
+----
+ext = Messagepack::ExtensionValue.new(0x20, "payload")
+binary = factory.pack(ext)
+----
+
+== Type ID registry
+
+This table tracks commonly used type IDs:
+
+[width="40%",options="header"]
+|===
+|Type ID |Type |Status
+|-128 to -1|Reserved|MessagePack specification
+|0|Symbol|Convention
+|1-127|Application|Available
+|===
+
+When creating custom extension types:
+* Check existing registrations for conflicts
+* Document your type IDs
+* Consider using a registry for distributed applications

data/docs/_references/format.adoc

@@ -0,0 +1,301 @@
+---
+title: Format specification
+nav_order: 2
+---
+
+== Purpose
+
+Detailed specification of the MessagePack binary format.
+
+== References
+
+* https://github.com/msgpack/msgpack/blob/master/spec.md[Official MessagePack specification]
+
+== Format overview
+
+MessagePack uses a compact binary format where the first byte contains both
+format information and often the value itself.
+
+== Integer types
+
+=== Positive fixint (0x00 - 0x7F)
+
+Stores integers 0-127 in a single byte.
+
+[source]
+----
++--------+
+|0XXXXXXX|
++--------+
+
+stores 0-127 in one byte
+----
+
+=== Negative fixint (0xE0 - 0xFF)
+
+Stores integers -1 to -32 in a single byte.
+
+[source]
+----
++--------+
+|111XXXXX|
++--------+
+
+stores -32 to -1 in one byte
+----
+
+=== uint8 (0xCC)
+
+Stores unsigned 8-bit integer.
+
+[source]
+----
++--------+--------+
+|  0xCC  |XXXXXXX|
++--------+--------+
+
+stores 0-255 in two bytes
+----
+
+=== uint16 (0xCD)
+
+Stores unsigned 16-bit integer (big-endian).
+
+[source]
+----
++--------+--------+--------+
+|  0xCD  |ZZZZZZZZ|ZZZZZZZZ|
++--------+--------+--------+
+
+stores 0-65535 in three bytes
+----
+
+=== uint32 (0xCE)
+
+Stores unsigned 32-bit integer (big-endian).
+
+[source]
+----
++--------+--------+--------+--------+--------+
+|  0xCE  |AAAAAAAA|AAAAAAAA|AAAAAAAA|AAAAAAAA|
++--------+--------+--------+--------+--------+
+
+stores 0-4294967295 in five bytes
+----
+
+=== uint64 (0xCF)
+
+Stores unsigned 64-bit integer (big-endian).
+
+[source]
+----
++--------+--------+--------+--------+--------+--------+--------+--------+--------+
+|  0xCF  |AAAAAAAA|AAAAAAAA|AAAAAAAA|AAAAAAAA|AAAAAAAA|AAAAAAAA|AAAAAAAA|AAAAAAAA|
++--------+--------+--------+--------+--------+--------+--------+--------+--------+
+
+stores 0-18446744073709551615 in nine bytes
+----
+
+=== int8 (0xD0)
+
+Stores signed 8-bit integer.
+
+[source]
+----
++--------+--------+
+|  0xD0  |ZZZZZZZZ|
++--------+--------+
+
+stores -128 to 127 in two bytes
+----
+
+=== int16 (0xD1)
+
+Stores signed 16-bit integer (big-endian).
+
+=== int32 (0xD2)
+
+Stores signed 32-bit integer (big-endian).
+
+=== int64 (0xD3)
+
+Stores signed 64-bit integer (big-endian).
+
+== Floating point types
+
+=== float32 (0xCA)
+
+Stores IEEE 754 single precision floating point (big-endian).
+
+[source]
+----
++--------+--------+--------+--------+--------+
+|  0xCA  |FFFFFFFF|FFFFFFFF|FFFFFFFF|FFFFFFFF|
++--------+--------+--------+--------+--------+
+
+stores float32 in five bytes
+----
+
+=== float64 (0xCB)
+
+Stores IEEE 754 double precision floating point (big-endian).
+
+[source]
+----
++--------+--------+--------+--------+--------+--------+--------+--------+--------+
+|  0xCB  |FFFFFFFF|FFFFFFFF|FFFFFFFF|FFFFFFFF|FFFFFFFF|FFFFFFFF|FFFFFFFF|FFFFFFFF|
++--------+--------+--------+--------+--------+--------+--------+--------+--------+
+
+stores float64 in nine bytes
+----
+
+== String types
+
+=== fixstr (0xA0 - 0xBF)
+
+Stores strings with length 0-31.
+
+[source]
+----
++--------+========+
+|101XXXXX|  data |
++--------+========+
+
+stores 0-31 byte strings in 2-32 bytes
+----
+
+=== str8 (0xD9)
+
+Stores strings with length 0-255.
+
+[source]
+----
++--------+--------+========+
+|  0xD9  |YYYYYYYY|  data |
++--------+--------+========+
+
+stores 0-255 byte strings in 2-257 bytes
+----
+
+=== str16 (0xDA)
+
+Stores strings with length 0-65535.
+
+[source]
+----
++--------+--------+--------+========+
+|  0xDA  |ZZZZZZZZ|ZZZZZZZZ|  data |
++--------+--------+--------+========+
+
+stores 0-65535 byte strings in 4-65539 bytes
+----
+
+=== str32 (0xDB)
+
+Stores strings with length 0-4294967295.
+
+== Binary types
+
+=== bin8 (0xC4)
+
+Stores binary data with length 0-255.
+
+=== bin16 (0xC5)
+
+Stores binary data with length 0-65535.
+
+=== bin32 (0xC6)
+
+Stores binary data with length 0-4294967295.
+
+== Array types
+
+=== fixarray (0x90 - 0x9F)
+
+Stores arrays with 0-15 elements.
+
+[source]
+----
++--------+~~~~~~~~~~~~~~~~~+
+|1001XXXX|    N objects    |
++--------+~~~~~~~~~~~~~~~~~+
+
+stores 0-15 element arrays
+----
+
+=== array16 (0xDC)
+
+Stores arrays with 0-65535 elements.
+
+=== array32 (0xDD)
+
+Stores arrays with 0-4294967295 elements.
+
+== Map types
+
+=== fixmap (0x80 - 0x8F)
+
+Stores maps with 0-15 key-value pairs.
+
+=== map16 (0xDE)
+
+Stores maps with 0-65535 key-value pairs.
+
+=== map32 (0xDF)
+
+Stores maps with 0-4294967295 key-value pairs.
+
+== Extension types
+
+=== fixext1 (0xD4)
+
+Stores extension with 1 byte data.
+
+=== fixext2 (0xD5)
+
+Stores extension with 2 bytes data.
+
+=== fixext4 (0xD6)
+
+Stores extension with 4 bytes data.
+
+=== fixext8 (0xD7)
+
+Stores extension with 8 bytes data.
+
+=== fixext16 (0xD8)
+
+Stores extension with 16 bytes data.
+
+=== ext8 (0xC7)
+
+Stores extension with length 0-255.
+
+=== ext16 (0xC8)
+
+Stores extension with length 0-65535.
+
+=== ext32 (0xC9)
+
+Stores extension with length 0-4294967295.
+
+== Reserved types
+
+=== nil (0xC0)
+
+Represents nil/null.
+
+[source]
+----
++--------+
+|  0xC0  |
++--------+
+----
+
+=== false (0xC2)
+
+Represents boolean false.
+
+=== true (0xC3)
+
+Represents boolean true.

data/docs/_references/index.adoc

@@ -0,0 +1,14 @@
+---
+title: References
+nav_order: 1
+---
+
+== Purpose
+
+References provide detailed technical specifications and API documentation.
+
+== References
+
+* link:../references/api[API reference] - Complete API documentation
+* link:../references/format[Format specification] - MessagePack binary format details
+* link:../references/extensions[Extension types reference] - Built-in and custom extension types

data/docs/_tutorials/extension-types.adoc

@@ -0,0 +1,170 @@
+---
+title: Custom extension types
+nav_order: 2
+---
+
+== Purpose
+
+This tutorial shows how to create custom extension types for serializing objects
+that don't have native MessagePack support.
+
+== References
+
+* link:../pages/extension-types[Extension types] - Extension type concepts
+* link:../references/extensions[Extension types reference] - Detailed documentation
+
+== Prerequisites
+
+* Completed link:../tutorials/getting-started[Getting started] tutorial
+
+== Understanding extension types
+
+MessagePack defines a set of built-in types (nil, boolean, integer, float, string,
+array, map). For custom Ruby classes, you need to register an extension type.
+
+An extension type requires:
+
+* A type ID (-128 to 127)
+* A packing method that converts your object to binary
+* An unpacking method that reconstructs your object from binary
+
+== Simple example: Point class
+
+=== Define the class
+
+[source,ruby]
+----
+class Point
+  attr_reader :x, :y
+
+  def initialize(x, y)
+    @x = x
+    @y = y
+  end
+
+  def to_msgpack_ext
+    # Pack as two 32-bit integers
+    [@x, @y].pack("ll")
+  end
+
+  def self.from_msgpack_ext(data)
+    x, y = data.unpack("ll")
+    new(x, y)
+  end
+
+  def to_s
+    "(#{@x}, #{@y})"
+  end
+end
+----
+
+=== Register with factory
+
+[source,ruby]
+----
+factory = Messagepack::Factory.new
+factory.register_type(0x01, Point,
+  packer: :to_msgpack_ext,
+  unpacker: :from_msgpack_ext
+)
+----
+
+=== Use it
+
+[source,ruby]
+----
+point = Point.new(10, 20)
+binary = factory.pack(point)
+result = factory.unpack(binary)
+puts result  # => "(10, 20)"
+----
+
+== Complex example: Nested objects
+
+=== Define a container class
+
+[source,ruby]
+----
+class TreeNode
+  attr_reader :value, :left, :right
+
+  def initialize(value, left = nil, right = nil)
+    @value = value
+    @left = left
+    @right = right
+  end
+
+  def to_msgpack_ext(packer)
+    # Serialize as hash with recursive packing
+    packer.write({
+      value: @value,
+      left: @left,
+      right: @right
+    })
+  end
+
+  def self.from_msgpack_ext(unpacker)
+    # Recursively deserialize
+    data = unpacker.read
+    new(
+      data[:value],
+      data[:left],
+      data[:right]
+    )
+  end
+end
+----
+
+=== Register as recursive
+
+[source,ruby]
+----
+factory = Messagepack::Factory.new
+factory.register_type(0x02, TreeNode,
+  packer: ->(obj, packer) { obj.to_msgpack_ext(packer) },
+  unpacker: ->(unpacker) { TreeNode.from_msgpack_ext(unpacker) },
+  recursive: true
+)
+----
+
+=== Use it
+
+[source,ruby]
+----
+tree = TreeNode.new(
+  1,
+  TreeNode.new(2),
+  TreeNode.new(3, TreeNode.new(4))
+)
+
+binary = factory.pack(tree)
+result = factory.unpack(binary)
+puts result.value  # => 1
+puts result.left.value  # => 2
+puts result.right.left.value  # => 4
+----
+
+== Using procs for simple types
+
+For simple classes, you can use procs directly:
+
+[source,ruby]
+----
+require 'uri'
+
+factory = Messagepack::Factory.new
+factory.register_type(0x10, URI,
+  packer: ->(uri) { uri.to_s },
+  unpacker: ->(str) { URI.parse(str) }
+)
+
+uri = URI.parse("https://example.com")
+binary = factory.pack(uri)
+result = factory.unpack(binary)
+# => #<URI::HTTPS https://example.com>
+----
+
+== Next steps
+
+* link:../tutorials/thread-safety[Thread-safe usage] - Use factories in multi-threaded applications
+* link:../guides/io-streaming[IO streaming] - Work with streams and networks