msgpack 0.4.7 → 0.5.0

data/.gitignore

@@ -0,0 +1,17 @@
+*.o
+*.so
+*.gem
+*.class
+.bundle
+Gemfile*
+pkg
+test/debug.log
+*~
+/rdoc
+tmp
+.classpath
+.project
+.settings
+/nbproject/private/
+ext/java/build
+ext/java/msgpack.jar

data/ChangeLog

@@ -0,0 +1,47 @@
+
+2012-12-20 version 0.5.0:
+
+* Rewrote all code and improved performance significantly
+* Added MessagePack::Buffer class
+* Added MessagePack::Packer class
+* Added Packer#buffer and Unpacker#buffer accessors which return MessagePack::Buffer
+* Added Packer#write_{array,map}_header and Unpacker#read_{array,map}_header methods
+* Added Packer#write_nil and Unpacker#skip_nil methods
+* Added Packer#write -> #pack alias and Unpacker#read method
+* Added exception classes - UnpackError, MalformedFormatError, StackError and TypeError
+* Added MessagePack.dup -> .pack and MessagePack.load -> .unpack aliases
+* Added Packer#empty?, #size and #clear methods
+* Added Packer#write_to(io) method to flush serialized data to IO efficiently
+* Added Unpacker#skip method to skip an object efficiently
+* Removed obsoleted Unpacker#fill, #execute, #execute_limit, #finished? and #data methods
+* Removed obsoleted Unapcker#stream and #stream= methods. Use unpacker.buffer.io instead
+
+
+2012-05-05 version 0.4.7:
+
+* Fixed serialization of double values on ARM OABI architectures
+* Fixed byteorder problem on big-endian platforms
+* Don't use MRI internals in the Ruby extension for Rubinius
+* Detect whether st.h is present and don't use RUBY_VM as the condition for
+  Rubinius
+
+2011-08-08 version 0.4.6:
+
+* Fixed compile error problem on Mac OS X Lion
+
+2011-05-09 version 0.4.5:
+
+* Improves compatibility with JRuby
+
+2010-11-28 version 0.4.4:
+
+* Adds Unpacker#feed_each method
+* Improves compatibility with Rubinius
+* Improves compatibility with ruby-1.8.5
+* Encodings of String instances to UTF-8 on Ruby 1.9
+
+2010-06-29 version 0.4.3:
+
+* Adds MessagePack::VERSION constant
+* Fixes SEGV problem caused by GC bug at MessagePack_Unpacker_mark
+

data/README.rdoc

@@ -0,0 +1,102 @@
+
+= MessagePack
+
+MessagePack[http://msgpack.org] is an efficient binary serialization format.
+It lets you exchange data among multiple languages like JSON but it's faster and smaller.
+For example, small integers (like flags or error code) are encoded into a single byte,
+and typical short strings only require an extra byte in addition to the strings themselves.
+
+If you ever wished to use JSON for convenience (storing an image with metadata) but could
+not for technical reasons (binary data, size, speed...), MessagePack is a perfect replacement.
+
+    require 'msgpack'
+    msg = [1,2,3].to_msgpack  #=> "\x93\x01\x02\x03"
+    MessagePack.unpack(msg)   #=> [1,2,3]
+
+Use RubyGems to install:
+
+    gem install msgpack
+
+
+= Use cases
+
+* Store objects efficiently serialized by msgpack on memcached or Redis
+  * In fact Redis supports msgpack in EVAL-scripts[http://redis.io/commands/eval]
+* Upload data in efficient format from mobile devices such as smartphones
+  * MessagePack works on iPhone/iPad and Android. See also Objective-C[https://github.com/msgpack/msgpack-objectivec] and Java[https://github.com/msgpack/msgpack-java] implementations
+* Design a portable protocol to communicate with embedded devices
+  * Check also Fluentd[http://fluentd.org/] which is a log collector which uses msgpack for the log format (they say it uses JSON but actually it's msgpack, which is compatible with JSON)
+* Exchange objects between software components written in different languages
+  * You'll need a flexible but efficient format so that components exchange objects while keeping compatibility
+
+
+= Portability
+
+MessagePack for Ruby should run on x86, ARM, PowerPC, SPARC and other CPU architectures.
+
+And it works with MRI (CRuby) and Rubinius.
+Patches to improve portability is highly welcomed.
+
+
+= Serializing objects
+
+Use *MessagePack.pack* or *to_msgpack*:
+
+    require 'msgpack'
+    msg = MessagePack.pack(obj)  # or
+    msg = obj.to_msgpack
+
+== Streaming serialization
+
+Packer provides advanced API to serialize objects in streaming style:
+
+    # serialize a 2-element array [e1, e2]
+    pk = MessagePack::Packer.new(io)
+    pk.write_array_header(2).write(e1).write(e2).flush
+
+See API reference for details.
+
+= Deserializing objects
+
+Use *MessagePack.unpack*:
+
+    require 'msgpack'
+    obj = MessagePack.unpack(msg)
+
+== Streaming deserialization
+
+Unpacker provides advanced API to deserialize objects in streaming style:
+
+    # deserialize objects from an IO
+    u = MessagePack::Unpacker.new(io)
+    u.each do |obj|
+      # ...
+    end
+
+or event-driven style which works well with EventMachine:
+
+    # event-driven deserialization
+    def on_read(data)
+      @u ||= MessagePack::Unpacker.new
+      @u.feed_each(data) {|obj|
+         # ...
+      }
+    end
+
+See API reference for details.
+
+
+= Buffer API
+
+MessagePack for Ruby provides a buffer API so that you can read or write data by hand, not via Packer or Unpacker API.
+
+This MessagePack::Buffer API is backed with a fixed-length shared memory pool which is very fast for small data (<= 4KB),
+and has zero-copy capability which significantly affects performance to handle large binary data.
+
+
+= Copyright
+
+Author::    FURUHASHI Sadayuki <frsyuki@gmail.com>
+Copyright:: Copyright (c) 2008-2012 FURUHASHI Sadayuki
+License::   Apache License, Version 2.0
+

data/Rakefile

@@ -0,0 +1,88 @@
+
+require 'bundler'
+Bundler::GemHelper.install_tasks
+
+require 'fileutils'
+
+require 'rspec/core'
+require 'rspec/core/rake_task'
+require 'yard'
+
+RSpec::Core::RakeTask.new(:spec) do |t|
+  t.rspec_opts = ["-c", "-f progress"]
+  t.rspec_opts << "-Ilib"
+  t.pattern = 'spec/**/*_spec.rb'
+  t.verbose = true
+end
+
+task :spec => :compile
+
+desc 'Run RSpec code examples and measure coverage'
+task :coverage do |t|
+  ENV['SIMPLE_COV'] = '1'
+  Rake::Task["spec"].invoke
+end
+
+desc 'Generate YARD document'
+YARD::Rake::YardocTask.new(:doc) do |t|
+  t.files   = ['lib/msgpack/version.rb','doclib/**/*.rb']
+  t.options = []
+  t.options << '--debug' << '--verbose' if $trace
+end
+
+spec = eval File.read("msgpack.gemspec")
+
+if RUBY_PLATFORM =~ /java/
+  require 'rake/javaextensiontask'
+
+  Rake::JavaExtensionTask.new('msgpack', spec) do |ext|
+    ext.ext_dir = 'ext/java'
+    #jruby_home = RbConfig::CONFIG['prefix']
+    #jars = ["#{jruby_home}/lib/jruby.jar"] + FileList['lib/*.jar']
+    #ext.classpath = jars.map { |x| File.expand_path x }.join ':'
+  end
+
+else
+  require 'rake/extensiontask'
+
+  Rake::ExtensionTask.new('msgpack', spec) do |ext|
+    ext.cross_compile = true
+    ext.lib_dir = File.join(*['lib', 'msgpack', ENV['FAT_DIR']].compact)
+    #ext.cross_platform = 'i386-mswin32'
+  end
+end
+
+task :default => :build
+
+
+###
+## Cross compile memo
+##
+## Ubuntu Ubuntu 10.04.1 LTS
+##
+#
+### install mingw32 cross compiler
+# sudo apt-get install gcc-mingw32
+#
+### install rbenv
+# git clone git://github.com/sstephenson/rbenv.git ~/.rbenv
+# echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
+# echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
+# exec $SHELL -l
+#
+### install cross-compiled ruby 1.9.3
+# rbenv install 1.9.3-p327
+# gem install rake-compiler
+# rake-compiler cross-ruby VERSION=1.9.3-p327
+#
+### install cross-compiled ruby 1.8.7
+# rbenv install 1.8.7-p371
+# gem install rake-compiler
+# rake-compiler cross-ruby VERSION=1.8.7-p371
+#
+### build gem
+# rbenv shell 1.8.7-p371
+# gem install bundler && bundle
+# rake cross native gem RUBY_CC_VERSION=1.8.7:1.9.3
+#
+

data/doclib/msgpack.rb

@@ -0,0 +1,55 @@
+
+module MessagePack
+  #
+  # Serializes an object into an IO or String.
+  #
+  # @overload dump(obj)
+  #   @return [String] serialized data
+  #
+  # @overload dump(obj, io)
+  #   @return [IO]
+  #
+  def self.dump(arg)
+  end
+
+  #
+  # Serializes an object into an IO or String. Alias of dump.
+  #
+  # @overload dump(obj)
+  #   @return [String] serialized data
+  #
+  # @overload dump(obj, io)
+  #   @return [IO]
+  #
+  def self.pack(arg)
+  end
+
+  #
+  # Deserializes an object from an IO or String.
+  #
+  # @overload load(string)
+  #   @param string [String] data to deserialize
+  #
+  # @overload load(io)
+  #   @param io [IO]
+  #
+  # @return [Object] deserialized object
+  #
+  def self.load(arg)
+  end
+
+  #
+  # Deserializes an object from an IO or String. Alias of load.
+  #
+  # @overload unpack(string)
+  #   @param string [String] data to deserialize
+  #
+  # @overload unpack(io)
+  #   @param io [IO]
+  #
+  # @return [Object] deserialized object
+  #
+  def self.unpack(arg)
+  end
+end
+

data/doclib/msgpack/buffer.rb

@@ -0,0 +1,193 @@
+module MessagePack
+
+  class Buffer
+    #
+    # Creates a MessagePack::Buffer instance.
+    #
+    # @overload initialize(options={})
+    #   @param options [Hash]
+    #
+    # @overload initialize(io, options={})
+    #   @param io [IO]
+    #   @param options [Hash]
+    #   This buffer writes written data into the IO when it is filled.
+    #   This buffer reads data from the IO when it is empty.
+    #
+    # _io_ must respond to readpartial(length, [,string]) or read(string) method and
+    # write(string) or append(string) method.
+    #
+    # Supported options:
+    #
+    # * *:io_buffer_size* buffer size to read data from the internal IO. (default: 32768)
+    # * *:read_reference_threshold* the threshold size to enable zero-copy deserialize optimization. Read strings longer than this threshold will refer the original string instead of copying it. (default: 256) (supported in MRI only)
+    # * *:write_reference_threshold* the threshold size to enable zero-copy serialize optimization. The buffer refers written strings longer than this threshold instead of copying it. (default: 524288) (supported in MRI only)
+    #
+    def initialize(*args)
+    end
+
+    #
+    # Makes the buffer empty
+    #
+    # @return nil
+    #
+    def clear
+    end
+
+    #
+    # Returns byte size of the buffer.
+    #
+    # @return nil
+    #
+    def size
+    end
+
+    #
+    # Returns _true_ if the buffer is empty.
+    # This method is slightly faster than _size_.
+    #
+    # @return [Boolean]
+    #
+    def empty?
+    end
+
+    #
+    # Appends the given data to the buffer.
+    #
+    # @param data [String]
+    # @return [Integer] byte size written
+    #
+    def write(data)
+    end
+
+    #
+    # Appends the given data to the buffer.
+    #
+    # @param data [String]
+    # @return [Buffer] self
+    #
+    def <<(data)
+    end
+
+    #
+    # Consumes _n_ bytes from the head of the buffer and returns consumed data.
+    # If the size of the buffer is less than _n_, it reads all of data in the buffer.
+    #
+    # If _n_ is 0, it does nothing and returns an empty string.
+    # If the optional _buffer_ argument is given, the content of the string will be replaced with the consumed data.
+    #
+    # @overload read
+    #
+    # @overload read(n)
+    #   @param n [Integer] bytes to read
+    #
+    # @overload read(n, buffer)
+    #   @param n [Integer] bytes to read
+    #   @param buffer [String] buffer to read into
+    #
+    # @return [String]
+    #
+    def read(n)
+    end
+
+    #
+    # Consumes _n_ bytes from the head of the buffer and returns consumed data.
+    # If the size of the buffer is less than _n_, it does nothing and raises EOFError.
+    #
+    # If _n_ is 0, it does nothing and returns an empty string.
+    # If the optional _buffer_ argument is given, the content of the string will be replaced with the consumed data.
+    #
+    # @overload read_all
+    #
+    # @overload read_all(n)
+    #   @param n [Integer] bytes to read
+    #
+    # @overload read_all(n, buffer)
+    #   @param n [Integer] bytes to read
+    #   @param buffer [String] buffer to read into
+    #
+    # @return [String]
+    #
+    def read_all(n, buffer=nil)
+    end
+
+    #
+    # Consumes _n_ bytes from the head of the buffer.
+    # If the size of the buffer is less than _n_, it skips all of data in the buffer and returns integer less than _n_.
+    #
+    # If _n_ is 0, it does nothing and returns _0_.
+    #
+    # @param n [Integer] byte size to skip
+    # @return [Integer] byte size actually skipped
+    #
+    def skip(n)
+    end
+
+    #
+    # Consumes _n_ bytes from the head of the buffer.
+    # If the size of the buffer is less than _n_, it does nothing and raises EOFError.
+    # If _n_ is 0, it does nothing.
+    #
+    # @param n [Integer] byte size to skip
+    # @return [Buffer] self
+    #
+    def skip_all(n)
+    end
+
+    #
+    # Returns all data in the buffer as a string.
+    # Destructive update to the returned string does NOT effect the buffer.
+    #
+    # @return [String]
+    #
+    def to_str
+    end
+
+    #
+    # Returns content of the buffer as an array of strings.
+    #
+    # This method is sometimes faster than to_s because the internal
+    # structure of the buffer is a queue of buffer chunks.
+    #
+    # @return [Array] array of strings
+    #
+    def to_a
+    end
+
+    #
+    # Internal io
+    #
+    # @return IO
+    #
+    attr_reader :io
+
+    #
+    # Flushes data in the internal buffer to the internal IO.
+    # If internal IO is not set, it does nothing.
+    #
+    # @return [Buffer] self
+    #
+    def flush
+    end
+
+    #
+    # Closes internal IO if its set.
+    # If internal IO is not set, it does nothing
+    #
+    # @return nil
+    #
+    def close
+    end
+
+    #
+    # Writes all of data in the internal buffer into the given IO.
+    # This method consumes and removes data from the internal buffer.
+    # _io_ must respond to write(data) method.
+    #
+    # @param io [IO]
+    # @return [Integer] byte size of written data
+    #
+    def write_to(io)
+    end
+  end
+
+end