messagepack 1.0.0

data/docs/_guides/performance.adoc

@@ -0,0 +1,189 @@
+---
+title: Performance optimization
+nav_order: 1
+---
+
+== Purpose
+
+This guide covers techniques for maximizing MessagePack Ruby performance.
+
+== References
+
+* link:../pages/buffer[Buffer management] - Buffer optimization details
+* link:../references/api[API reference] - Low-level API documentation
+
+== Concepts
+
+=== Native type fast-path
+
+Native MessagePack types bypass the extension registry:
+
+* nil, boolean, integer, float, string, symbol, array, hash
+* No O(n) registry search for these types
+* Performance unaffected by number of registered extension types
+
+=== Buffer coalescing
+
+Small writes are automatically merged:
+
+* Writes < 512 bytes are coalesced into larger chunks
+* Reduces memory allocations and GC pressure
+* Significantly faster for patterns with many small writes
+
+=== Factory pools
+
+Reuse packer/unpacker instances:
+
+* Avoid allocation overhead
+* Thread-safe concurrent access
+* Configure pool size for your workload
+
+== Optimization techniques
+
+=== Use native types when possible
+
+[source,ruby]
+----
+# Faster: Native hash
+data = { name: "Alice", age: 30 }
+
+# Slower: Custom class requires registry lookup
+class Person
+  attr_accessor :name, :age
+end
+data = Person.new("Alice", 30)
+----
+
+=== Use factory pools for repeated operations
+
+[source,ruby]
+----
+# Create a pool for repeated use
+pool = factory.pool(10)
+
+# Fast: Reuses packer from pool
+1000.times do |i|
+  pool.pack(id: i, value: "data")
+end
+----
+
+=== Use streaming for large data
+
+[source,ruby]
+----
+# Memory efficient for large files
+buffer = Messagepack::BinaryBuffer.new(File.open("large.msgpack", "rb"))
+unpacker = Messagepack::Unpacker.new(buffer)
+
+while obj = unpacker.read
+  process(obj)  # One object at a time
+end
+----
+
+=== Minimize extension type overhead
+
+[source,ruby]
+----
+# Faster: Use native types in hash
+data = { type: "user", id: 123 }
+
+# Slower: Custom type class requires registry lookup
+data = DataWrapper.new("user", 123)
+----
+
+== Benchmarking
+
+=== Simple benchmark
+
+[source,ruby]
+----
+require 'benchmark'
+
+data = { key: "value" * 100 }
+
+n = 10000
+
+Benchmark.bm do |x|
+  x.report("pack:") do
+    n.times { Messagepack.pack(data) }
+  end
+
+  x.report("unpack:") do
+    binary = Messagepack.pack(data)
+    n.times { Messagepack.unpack(binary) }
+  end
+end
+----
+
+=== Comparing approaches
+
+[source,ruby]
+----
+# Compare factory vs pool
+factory = Messagepack::Factory.new
+pool = factory.pool(5)
+
+Benchmark.bm do |x|
+  x.report("factory:") do
+    1000.times { factory.pack(data) }
+  end
+
+  x.report("pool:") do
+    1000.times { pool.pack(data) }
+  end
+end
+----
+
+== Performance characteristics
+
+Typical performance on modern hardware:
+
+* Packing simple values: 500k-700k ops/sec
+* Packing small arrays: 200k-300k ops/sec
+* Packing small hashes: 150k-200k ops/sec
+* Buffer operations: ~30% faster with coalescing
+
+Your results will vary based on:
+* Data size and complexity
+* Ruby implementation (MRI, JRuby, TruffleRuby)
+* Hardware and OS
+* Number of registered extension types
+
+== Common pitfalls
+
+=== Creating new factories repeatedly
+
+[source,ruby]
+----
+# Bad: Creates new factory each time
+def process(data)
+  factory = Messagepack::Factory.new
+  factory.pack(data)
+end
+
+# Good: Reuse factory
+FACTORY = Messagepack::Factory.new.freeze
+
+def process(data)
+  FACTORY.pack(data)
+end
+----
+
+=== Not using pools in threads
+
+[source,ruby]
+----
+# Bad: Shared unpacker is not thread-safe
+UNPACKER = Messagepack::Unpacker.new
+
+threads.map do
+  Thread.new { UNPACKER.unpack(data) }
+end
+
+# Good: Use pool
+pool = factory.pool(10)
+
+threads.map do
+  Thread.new { pool.unpack(data) }
+end
+----

data/docs/_pages/buffer.adoc

@@ -0,0 +1,85 @@
+---
+title: Buffer
+parent: Core topics
+nav_order: 6
+---
+
+[[buffer]]
+= Buffer management
+
+== Purpose
+
+The `BinaryBuffer` class provides efficient chunked storage for binary data.
+
+== References
+
+* link:../references/api[API reference] - BinaryBuffer class documentation
+* link:../guides/performance[Performance guide] - Buffer optimization techniques
+
+== Concepts
+
+=== Buffer operations
+
+[source,ruby]
+----
+buffer = Messagepack::BinaryBuffer.new
+buffer << "data"
+buffer.read(4)             # => "data"
+buffer.to_s                # => ""
+----
+
+Where,
+
+* `BinaryBuffer.new` creates a new buffer
+* `<<` appends data to the buffer
+* `read(n)` reads and consumes n bytes
+* `to_s` returns remaining data without consuming
+
+=== Skip operations
+
+[source,ruby]
+----
+buffer = Messagepack::BinaryBuffer.new
+buffer << "\x81\xA3foo\xA5world"
+buffer.skip              # Skip one object (format byte)
+buffer.skip_nil          # Skip nil value if present
+----
+
+Where,
+
+* `skip` skips a complete MessagePack object
+* `skip_nil` efficiently skips nil values
+
+=== Buffer with IO
+
+[source,ruby]
+----
+File.open("data.msgpack", "rb") do |io|
+  buffer = Messagepack::BinaryBuffer.new(io)
+  unpacker = Messagepack::Unpacker.new(buffer)
+  obj = unpacker.read
+end
+----
+
+Where,
+
+* The buffer reads from the IO when needed
+* Data is automatically managed in chunks
+* Suitable for large files that don't fit in memory
+
+== Examples
+
+.Reading large MessagePack files efficiently
+====
+[source,ruby]
+----
+# Process a large file without loading everything into memory
+buffer = Messagepack::BinaryBuffer.new(File.open("large.msgpack", "rb"))
+unpacker = Messagepack::Unpacker.new(buffer)
+
+while obj = unpacker.read
+  # Process each object one at a time
+  process(obj)
+end
+----
+====

data/docs/_pages/extension-types.adoc

@@ -0,0 +1,117 @@
+---
+title: Extension types
+parent: Core topics
+nav_order: 3
+---
+
+[[extension-types]]
+= Extension types
+
+== Purpose
+
+MessagePack supports custom extension types for serializing objects that don't
+have a native MessagePack representation.
+
+== References
+
+* link:../references/extensions[Extension types reference] - Detailed extension documentation
+* link:../tutorials/extension-types[Custom extension types tutorial] - Step-by-step guide
+
+== Concepts
+
+=== Extension type format
+
+[source,ruby]
+----
+factory.register_type(type_id, class,
+  packer: packer_specification,
+  unpacker: unpacker_specification
+)
+----
+
+Where,
+
+* `type_id` is an integer from -128 to 127
+* `class` is the Ruby class to serialize
+* `packer_specification` can be:
+  ** A symbol (method name to call on the object)
+  ** A proc (called with the object)
+  ** A method object
+* `unpacker_specification` can be:
+  ** A symbol (class method to call)
+  ** A proc (called with the payload data)
+  ** A method object
+
+=== Recursive extension types
+
+[source,ruby]
+----
+factory.register_type(0x02, MyContainer,
+  packer: ->(obj, packer) { packer.write(obj.to_h) },
+  unpacker: ->(unpacker) { MyContainer.from_hash(unpacker.read) },
+  recursive: true
+)
+----
+
+Where,
+
+* `recursive: true` enables nested serialization
+* The `packer` lambda receives the packer instance for recursive calls
+* The `unpacker` lambda receives the unpacker instance for recursive reads
+
+== Examples
+
+.Custom extension type for Money objects
+====
+[source,ruby]
+----
+class Money
+  attr_reader :amount, :currency
+
+  def initialize(amount, currency)
+    @amount = amount
+    @currency = currency
+  end
+
+  def to_msgpack_ext
+    [amount, currency].pack("QA*")
+  end
+
+  def self.from_msgpack_ext(data)
+    amount, currency = data.unpack("QA*")
+    new(amount, currency)
+  end
+end
+
+factory = Messagepack::Factory.new
+factory.register_type(0x10, Money,
+  packer: :to_msgpack_ext,
+  unpacker: :from_msgpack_ext
+)
+
+money = Money.new(1000, "USD")
+binary = factory.pack(money)
+result = factory.unpack(binary)
+# => #<Money:0x... @amount=1000, @currency="USD">
+----
+====
+
+.Using procs for simple serialization
+====
+[source,ruby]
+----
+factory = Messagepack::Factory.new
+
+# Serialize a URI object
+require 'uri'
+factory.register_type(0x20, URI,
+  packer: ->(uri) { uri.to_s },
+  unpacker: ->(str) { URI.parse(str) }
+)
+
+uri = URI.parse("https://example.com/path")
+binary = factory.pack(uri)
+result = factory.unpack(binary)
+# => #<URI::HTTPS https://example.com/path>
+----
+====

data/docs/_pages/factory-pattern.adoc

@@ -0,0 +1,115 @@
+---
+title: Factory pattern
+parent: Core topics
+nav_order: 2
+---
+
+[[factory-pattern]]
+= Factory pattern
+
+== Purpose
+
+The `Messagepack::Factory` class provides thread-safe management of packer and
+unpacker instances with support for custom type registrations.
+
+== References
+
+* link:../references/api[API reference] - Factory class documentation
+* link:../guides/thread-safety[Thread-safe usage tutorial] - Multi-threading best practices
+
+== Concepts
+
+=== Creating a factory
+
+[source,ruby]
+----
+factory = Messagepack::Factory.new
+----
+
+Where,
+
+* `Factory.new` creates a new factory instance
+* Each factory maintains its own type registry
+* Factories can be frozen for thread-safe use
+
+=== Registering custom types
+
+[source,ruby]
+----
+factory.register_type(0x01, MyClass,
+  packer: :to_msgpack_ext,
+  unpacker: :from_msgpack_ext
+)
+----
+
+Where,
+
+* `0x01` is the type identifier (must be -128 to 127)
+* `MyClass` is the Ruby class to register
+* `packer` specifies how to serialize instances (symbol, method, or proc)
+* `unpacker` specifies how to deserialize data (symbol, method, or proc)
+
+=== Using factory pool for thread safety
+
+[source,ruby]
+----
+pool = factory.pool(5) # Create pool with 5 packers/unpackers
+data = pool.pack(my_object)  # Thread-safe packing
+obj = pool.unpack(binary)    # Thread-safe unpacking
+----
+
+Where,
+
+* `factory.pool(size)` creates a thread-safe pool
+* `size` is the number of packer/unpacker instances in the pool
+* The pool automatically manages instance reuse
+* Each thread gets its own instance from the pool
+
+== Examples
+
+.Thread-safe factory usage
+====
+[source,ruby]
+----
+# Create a factory with custom types
+factory = Messagepack::Factory.new
+factory.register_type(0x01, MyCustomClass,
+  packer: ->(obj) { obj.serialize },
+  unpacker: ->(data) { MyCustomClass.deserialize(data) }
+)
+
+# Create a thread-safe pool
+pool = factory.pool(10)
+
+# Use from multiple threads safely
+threads = 10.times.map do |i|
+  Thread.new do
+    object = MyCustomClass.new("data-#{i}")
+    binary = pool.pack(object)
+    result = pool.unpack(binary)
+    result.value
+  end
+end
+
+puts threads.map(&:value).inspect
+----
+====
+
+.Pool size and performance
+====
+[source,ruby]
+----
+# Create factory
+factory = Messagepack::Factory.new
+
+# Pool size should match expected concurrent threads
+# Too small: threads wait for available instances
+# Too much: wasted memory
+
+pool = factory.pool(4)  # Match your thread pool size
+
+# Default factory is also available
+Messagepack.pack(data)   # Uses DefaultFactory internally
+Messagepack::DefaultFactory.register_type(0x01, MyClass, ...)
+----
+====

data/docs/_pages/index.adoc

@@ -0,0 +1,20 @@
+---
+title: Core topics
+nav_order: 1
+parent: Core topics
+grand_parent: Core topics
+---
+
+== Purpose
+
+Core topics cover fundamental concepts and essential information for using the MessagePack Ruby library.
+
+== Topics
+
+* link:../serialization[Serialization] - Converting Ruby objects to binary format and back
+* link:../factory-pattern[Factory pattern] - Thread-safe packer/unpacker management
+* link:../extension-types[Extension types] - Custom type registration system
+* link:../timestamp-extension[Timestamp extension] - Nanosecond precision time handling
+* link:../symbol-extension[Symbol extension] - Efficient symbol serialization
+* link:../buffer[Buffer management] - Efficient chunked binary data storage
+* link:../streaming[Streaming] - Incremental data parsing

data/docs/_pages/serialization.adoc

@@ -0,0 +1,159 @@
+---
+title: Serialization
+parent: Core topics
+nav_order: 1
+---
+
+[[serialization]]
+= Serialization
+
+== Purpose
+
+The core MessagePack API provides simple `pack` and `unpack` methods for
+serializing and deserializing Ruby objects to and from binary format.
+
+== References
+
+* link:../references/api[API reference] - Complete method documentation
+* link:../references/format[Format specification] - Binary format details
+
+== Concepts
+
+=== Packing objects
+
+Use `Messagepack.pack` to serialize Ruby objects to binary format:
+
+[source,ruby]
+----
+Messagepack.pack({hello: "world"}) # => "\x81\xA5hello\xA5world"
+----
+
+Where,
+
+* `Messagepack.pack` accepts any Ruby object as its argument
+* The return value is a binary string containing the serialized data
+* Supported types include: nil, boolean, integer, float, string, array,
+  hash, and any registered extension types
+
+=== Unpacking data
+
+Use `Messagepack.unpack` to deserialize binary data back to Ruby objects:
+
+[source,ruby]
+----
+data = Messagepack.pack({hello: "world"})
+Messagepack.unpack(data) # => {"hello"=>"world"}
+----
+
+Where,
+
+* `Messagepack.unpack` accepts a binary string or IO object
+* The return value is the original Ruby object
+* Extra bytes after the deserialized object will raise a
+  `Messagepack::MalformedFormatError`
+
+== Supported types
+
+=== Nil
+
+[source,ruby]
+----
+Messagepack.pack(nil) # => "\xC0"
+Messagepack.unpack("\xC0") # => nil
+----
+
+=== Boolean
+
+[source,ruby]
+----
+Messagepack.pack(true)  # => "\xC3"
+Messagepack.pack(false) # => "\xC2"
+Messagepack.unpack("\xC3") # => true
+Messagepack.unpack("\xC2") # => false
+----
+
+=== Integer
+
+MessagePack supports integers from -2^63 to 2^64-1:
+
+[source,ruby]
+----
+Messagepack.pack(0)   # => "\x00"
+Messagepack.pack(127) # => "\x7F"
+Messagepack.pack(128) # => "\xCC\x80"
+Messagepack.pack(-1)  # => "\xFF"
+Messagepack.unpack("\xCC\x80") # => 128
+----
+
+=== Float
+
+[source,ruby]
+----
+Messagepack.pack(3.14) # => "\xCB@\x09\x1E\xB8Q\xEB\x85\x1F"
+Messagepack.unpack("\xCB@\x09\x1E\xB8Q\xEB\x85\x1F") # => 3.14
+----
+
+=== String
+
+Strings are encoded as UTF-8:
+
+[source,ruby]
+----
+Messagepack.pack("hello") # => "\xA5hello"
+Messagepack.unpack("\xA5hello") # => "hello"
+----
+
+=== Array
+
+[source,ruby]
+----
+Messagepack.pack([1, 2, 3]) # => "\x93\x01\x02\x03"
+Messagepack.unpack("\x93\x01\x02\x03") # => [1, 2, 3]
+----
+
+=== Hash
+
+[source,ruby]
+----
+Messagepack.pack({a: 1, b: 2}) # => "\x82\xA1a\x01\xA1b\x02"
+Messagepack.unpack("\x82\xA1a\x01\xA1b\x02") # => {"a"=>1, "b"=>2}
+----
+
+== Examples
+
+.Using pack and unpack
+====
+[source,ruby]
+----
+# Serialize a complex object
+data = {
+  name: "Alice",
+  age: 30,
+  skills: ["Ruby", "Python"],
+  metadata: {
+    active: true,
+    score: 95.5
+  }
+}
+
+binary = Messagepack.pack(data)
+# => "\x84\xA4name\xA5Alice\xA3age\x1E\xA6skills\
+#     \x92\xA4Ruby\xA6Python\xA8metadata\x82\xA6active\
+#     \xC3\xA5score\xCB@_\x00\x00"
+
+# Deserialize back to a Ruby object
+result = Messagepack.unpack(binary)
+# => {"name"=>"Alice", "age"=>30, "skills"=>["Ruby", "Python"],
+#     "metadata"=>{"active"=>true, "score"=>95.5}}
+----
+====
+
+== Aliases
+
+The module also provides `dump` and `load` as aliases:
+
+[source,ruby]
+----
+Messagepack.dump(data)  # Same as Messagepack.pack(data)
+Messagepack.load(data)  # Same as Messagepack.unpack(data)
+----