rbzip2 0.2.0 → 0.3.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 5211bc090f23faf0813dcea8e9c404e4cf178823
+  data.tar.gz: 5694495d649a64221b17a89cf9d0c27527cd955e
+SHA512:
+  metadata.gz: 171681ece3ad8fa0266f4754e3cfff9be61fcc82d447d01d0195d7a12d62e2cccabb4b43806922fb37fee06aafef1a0488c91259cdb2833deeb0bd692da4e6b4
+  data.tar.gz: d17e1d0989d1105f38709d9a6b941729e63bef120764c6262f879228df852a963ea485d0376979d781eb4534c68d6d584f471dc6ee7c4233852233bbee47a4d9

data/LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2011, Sebastian Staudt
+Copyright (c) 2011-2017, Sebastian Staudt
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification,

data/README.md

@@ -1,37 +1,54 @@
 RBzip2
 ======
 
-RBzip2 is a gem providing a pure Ruby implementation of the [bzip2][1]
-algorithm used for compression and decompression.
+RBzip2 is a gem providing various implementations of the [bzip2][bzip2]
+algorithm used for compression and decompression. Currently, it includes a
+[FFI][ffi]-based implementation and a pure Ruby implementation that's slower
+but works on any Ruby VM. Additionally, there's a JRuby specific implementation
+that's based on Commons Compress.
 
-It is based on the code of the [Apache Commons Compress][2] project and adds
-a straight Ruby-like API. There are no external dependencies like other gems or
-libraries. Therefore it will run on any Ruby implementation and the respective
-operating systems supported by those implementations.
+The pure Ruby implementations is based on the code of the
+[Apache Commons Compress][commons] project and adds a straight Ruby-like API.
+There are no external dependencies like other gems or libraries. Therefore it
+will run on any Ruby implementation and the respective operating systems
+supported by those implementations.
+
+The FFI implementation is using `libbz2` and provides fast performance on
+platforms where both `libbz2` and FFI are available. It is derived from this
+[Gist by Brian Lopez][gist].
+
+The Java-based implementation can use the
+[Commons Compress Java library][commons] if it is available in the classpath.
 
 ## Features
 
- * Compression of raw data into bzip2 compressed `IO`s (like `File` or
-   `StringIO`)
- * Decompression of bzip2 compressed `IO`s (like `File` or `StringIO`)
+ * Compression of raw data into bzip2 compressed `IO`s (like `File` or
+   `StringIO`)
+ * Decompression of bzip2 compressed `IO`s (like `File` or `StringIO`)
 
 ## Usage
 
-    require 'rbzip2'
-
-### Compression
-
-    data = some_data
-    file = File.new 'somefile.bz2'      # open the target file
-    bz2  = RBzip2::Compressor.new file  # wrap the file into the compressor
-    bz2.write data                      # write the raw data to the compressor
-    bz2.close                           # finish compression (important!)
-
-### Decompression
-
-    file = File.new 'somefile.bz2'        # open a compressed file
-    bz2  = RBzip2::Decompressor.new file  # wrap the file into the decompressor
-    data = io.read                        # read data into a string
+```ruby
+require 'rbzip2'
+```
+
+### Compression
+
+```ruby
+data = some_data
+file = File.new 'somefile.bz2'      # open the target file
+bz2  = RBzip2::Compressor.new file  # wrap the file into the compressor
+bz2.write data                      # write the raw data to the compressor
+bz2.close                           # finish compression (important!)
+```
+
+### Decompression
+
+```ruby
+file = File.new 'somefile.bz2'        # open a compressed file
+bz2  = RBzip2::Decompressor.new file  # wrap the file into the decompressor
+data = io.read                        # read data into a string
+```
 
 ## Future plans
 
@@ -43,21 +60,37 @@ operating systems supported by those implementations.
 
 To install RBzip2 as a Ruby gem use the following command:
 
-    gem install rbzip2
+```sh
+$ gem install rbzip2
+```
 
 To use it as a dependency managed by Bundler add the following to your
 `Gemfile`:
 
-    gem 'rbzip2'
+```ruby
+gem 'rbzip2'
+```
+
+If you want to use the FFI implementation on any non-JRuby VM, be sure to also
+install the `ffi` gem.
+
+## Performance
+
+The `bzip2-ruby` gem is a Ruby binding to `libbz2` and offers best performance,
+but it is only available for MRI < 2.0.0 and Rubinius.
+
+The FFI implementation binds to `libbz2` as well and has almost the same
+performance as `bzip2-ruby`.
+
+The Java implementation uses a native Java library and is slower by a factor of
+about 2/10 while compressing/decompressing.
+
+The pure Ruby implementation of RBzip2 is inherently slower than `bzip2-ruby`.
+Currently, this is a plain port of Apache Commons' Java code to Ruby and no
+effort has been made to optimize it. That's why the Ruby implementation of
+RBzip2 is slower by a factor of about 130/100 while compressing/decompressing
+(on Ruby 1.9.3). Ruby 1.8.7 is even slower.
 
-## Performance
-
-Due to its pure Ruby implementation RBzip2 is inherently slower than
-bzip2-ruby, which is a Ruby binding to libbzip2. Currently, RBzip2 is a plain
-port of Apache Commons' Java code to Ruby and no effort has been made to
-optimize it. That's why RBzip2 is slower by a factor of about 140/1000 while
-compressing/decompressing (on Ruby 1.9.3). Ruby 1.8.7 is even slower.
-
 ## License
 
 This code is free software; you can redistribute it and/or modify it under the
@@ -67,12 +100,15 @@ included LICENSE file.
 ## Credits
 
 * Sebastian Staudt -- koraktor(at)gmail.com
+* Brian Lopez -- seniorlopez(at)gmail.com
 
 ## See Also
 
 * [Documentation](http://rubydoc.info/gems/rbzip2)
 * [GitHub project page](https://github.com/koraktor/rbzip2)
-* [bzip2 project page][1]
+* [bzip2 project page][bzip2]
 
- [1]: http://bzip.org
- [2]: http://commons.apache.org/compress
+ [bzip2]:   http://bzip.org
+ [commons]: http://commons.apache.org/compress
+ [ffi]:     https://github.com/ffi/ffi/wiki
+ [gist]:    https://gist.github.com/brianmario/5833373

data/Rakefile

@@ -5,10 +5,12 @@
 #
 # Copyright (c) 2011, Sebastian Staudt
 
+require 'bundler/gem_tasks'
 require 'rspec/core/rake_task'
 require 'rubygems/package_task'
 
 task :default => :spec
+task :test    => :spec
 
 spec = Gem::Specification.load 'rbzip2.gemspec'
 Gem::PackageTask.new(spec) do |pkg|

data/lib/core_ext/io.rb

@@ -0,0 +1,12 @@
+# This code is free software; you can redistribute it and/or modify it under
+# the terms of the new BSD License.
+#
+# Copyright (c) 2011-2013, Sebastian Staudt
+
+unless IO.method_defined? :readbyte
+
+  def IO.readbyte
+    read(1)[0].ord
+  end
+
+end

data/lib/rbzip2.rb

@@ -1,17 +1,21 @@
 # This code is free software; you can redistribute it and/or modify it under
 # the terms of the new BSD License.
 #
-# Copyright (c) 2011, Sebastian Staudt
+# Copyright (c) 2011-2013, Sebastian Staudt
 
 module RBzip2
 
-  autoload :CRC,          'rbzip2/crc'
-  autoload :Compressor,   'rbzip2/compressor'
-  autoload :Constants,    'rbzip2/constants'
-  autoload :Decompressor, 'rbzip2/decompressor'
-  autoload :IO,           'rbzip2/io'
-  autoload :InputData,    'rbzip2/input_data'
-  autoload :OutputData,   'rbzip2/output_data'
-  autoload :VERSION,      'rbzip2/version'
+  autoload :Adapter, 'rbzip2/adapter'
+  autoload :FFI,     'rbzip2/ffi'
+  autoload :IO,      'rbzip2/io'
+  autoload :Java,    'rbzip2/java'
+  autoload :Ruby,    'rbzip2/ruby'
+  autoload :VERSION, 'rbzip2/version'
+
+  def self.default_adapter
+    return FFI if FFI.available?
+    return Java if Java.available?
+    Ruby
+  end
 
 end

data/lib/rbzip2/adapter.rb

@@ -0,0 +1,17 @@
+# This code is free software; you can redistribute it and/or modify it under
+# the terms of the new BSD License.
+#
+# Copyright (c) 2013, Sebastian Staudt
+
+module RBzip2::Adapter
+
+  def self.extended(mod)
+    mod.send :class_variable_set, :@@available, true
+    mod.init if mod.respond_to? :init
+  end
+
+  def available?
+    class_variable_get :@@available
+  end
+
+end

data/lib/rbzip2/ffi.rb

@@ -0,0 +1,33 @@
+# This code is free software; you can redistribute it and/or modify it under
+# the terms of the new BSD License.
+#
+# Copyright (c) 2013, Sebastian Staudt
+
+begin
+  require 'ffi'
+rescue LoadError
+end
+
+module RBzip2::FFI
+
+  def self.init
+    begin
+      extend ::FFI::Library
+      ffi_lib 'bz2'
+    rescue NameError, LoadError
+      @@available = false
+    end
+  end
+
+  extend RBzip2::Adapter
+
+  autoload :BufferError,  'rbzip2/ffi/errors'
+  autoload :Compressor,   'rbzip2/ffi/compressor'
+  autoload :ConfigError,  'rbzip2/ffi/errors'
+  autoload :CorruptError, 'rbzip2/ffi/errors'
+  autoload :Decompressor, 'rbzip2/ffi/decompressor'
+  autoload :Error,        'rbzip2/ffi/errors'
+
+end
+
+require 'rbzip2/ffi/constants'

data/lib/rbzip2/ffi/compressor.rb

@@ -0,0 +1,85 @@
+# This code is free software; you can redistribute it and/or modify it under
+# the terms of the new BSD License.
+#
+# Copyright (c) 2013, Brian Lopez
+# Copyright (c) 2013-2017, Sebastian Staudt
+
+class RBzip2::FFI::Compressor
+
+  extend ::FFI::Library
+
+  ffi_lib 'bz2'
+  attach_function :BZ2_bzBuffToBuffCompress,
+                  [:pointer, :buffer_inout, :pointer, :uint32, :int, :int, :int],
+                  :int
+
+  def self.compress(data, blksize = RBzip2::FFI::DEFAULT_BLK_SIZE, verbosity = 0, work_factor = 30)
+    blksize = 1 if blksize < 1
+    blksize = 9 if blksize > 9
+    verbosity = 0 if verbosity < 0
+    verbosity = 4 if verbosity > 4
+    work_factor = 0 if work_factor < 0
+    work_factor = 250 if work_factor > 250
+
+    out_len = data.bytesize + (data.bytesize * 0.01) + 600
+    dst_buf = ::FFI::MemoryPointer.new :char, out_len
+    dst_len = ::FFI::MemoryPointer.new :uint32
+    dst_len.write_uint out_len
+
+    src_buf = ::FFI::MemoryPointer.new :char, data.bytesize
+    src_buf.put_bytes 0, data
+
+    ret = BZ2_bzBuffToBuffCompress dst_buf, dst_len, src_buf, data.bytesize,
+                                   blksize, verbosity, work_factor
+
+    case ret
+      when RBzip2::FFI::BZ_OK
+        dst_buf.read_bytes dst_len.read_uint
+      when RBzip2::FFI::BZ_PARAM_ERROR
+        raise ArgumentError, 'One of blksize, verbosity or work_factor is out of range'
+      when RBzip2::FFI::BZ_MEM_ERROR
+        raise NoMemoryError, 'Out of memory'
+      when RBzip2::FFI::BZ_OUTBUFF_FULL
+        raise RBzip2::FFI::BufferError, "Output buffer isn't large enough"
+      when RBzip2::FFI::BZ_CONFIG_ERROR
+        raise RBzip2::FFI::ConfigError, 'libbz2 has been mis-compiled'
+      else
+        raise RBzip2::FFI::Error, "Unhandled error code: #{ret}"
+    end
+  end
+
+  def initialize(io)
+    @io = io
+  end
+
+  def flush
+    @io.flush unless @io.nil?
+  end
+
+  def close
+    flush
+    unless @io.nil?
+      @io.close
+      @io = nil
+    end
+  end
+
+  def putc(int)
+    if int.is_a? Numeric
+      write int & 0xff
+    else
+      write int.to_s[0].chr
+    end
+  end
+
+  def puts(line)
+    write line + $/
+  end
+
+  def write(bytes)
+    raise 'stream closed' if @io.nil?
+
+    @io.write self.class.compress(bytes, 9)
+  end
+
+end

data/lib/rbzip2/ffi/constants.rb

@@ -0,0 +1,30 @@
+# This code is free software; you can redistribute it and/or modify it under
+# the terms of the new BSD License.
+#
+# Copyright (c) 2013, Brian Lopez
+# Copyright (c) 2013, Sebastian Staudt
+
+module RBzip2::FFI
+
+  DEFAULT_BLK_SIZE    = 3
+
+  BZ_RUN              = 0
+  BZ_FLUSH            = 1
+  BZ_FINISH           = 2
+
+  BZ_OK               = 0
+  BZ_RUN_OK           = 1
+  BZ_FLUSH_OK         = 2
+  BZ_FINISH_OK        = 3
+  BZ_STREAM_END       = 4
+  BZ_SEQUENCE_ERROR   = -1
+  BZ_PARAM_ERROR      = -2
+  BZ_MEM_ERROR        = -3
+  BZ_DATA_ERROR       = -4
+  BZ_DATA_ERROR_MAGIC = -5
+  BZ_IO_ERROR         = -6
+  BZ_UNEXPECTED_EOF   = -7
+  BZ_OUTBUFF_FULL     = -8
+  BZ_CONFIG_ERROR     = -9
+
+end

data/lib/rbzip2/ffi/decompressor.rb

@@ -0,0 +1,163 @@
+# This code is free software; you can redistribute it and/or modify it under
+# the terms of the new BSD License.
+#
+# Copyright (c) 2013, Brian Lopez
+# Copyright (c) 2013-2017, Sebastian Staudt
+
+class RBzip2::FFI::Decompressor
+
+  extend ::FFI::Library
+
+  ffi_lib ::FFI::Platform::LIBC
+  attach_function :fopen,
+                  [:string, :string],
+                  :pointer
+  attach_function :fclose,
+                  [:pointer],
+                  :int
+
+  ffi_lib 'bz2'
+  attach_function :BZ2_bzRead,
+                  [:pointer, :pointer, :pointer, :int],
+                  :int
+  attach_function :BZ2_bzReadClose,
+                  [:pointer, :pointer],
+                  :void
+  attach_function :BZ2_bzReadOpen,
+                  [:pointer, :pointer, :int, :int, :pointer, :int],
+                  :pointer
+  attach_function :BZ2_bzBuffToBuffDecompress,
+                  [:pointer, :buffer_inout, :pointer, :uint32, :int, :int],
+                  :int
+
+  def self.decompress(data, factor = 2, small = 0, verbosity = 0)
+    out_len = data.bytesize * factor
+    dst_buf = ::FFI::MemoryPointer.new :char, out_len
+    dst_len = ::FFI::MemoryPointer.new :uint32
+    dst_len.write_uint out_len
+
+    src_buf = ::FFI::MemoryPointer.new :char, data.bytesize
+    src_buf.put_bytes 0, data
+
+    ret = BZ2_bzBuffToBuffDecompress dst_buf, dst_len, src_buf, data.bytesize,
+                                     small, verbosity
+
+    case ret
+      when RBzip2::FFI::BZ_OK
+        dst_buf.read_bytes dst_len.read_uint
+      when RBzip2::FFI::BZ_PARAM_ERROR
+        raise ArgumentError, 'One of sall or verbosity'
+      when RBzip2::FFI::BZ_MEM_ERROR
+        raise NoMemoryError, 'Out of memory'
+      when RBzip2::FFI::BZ_OUTBUFF_FULL
+        raise RBzip2::FFI::BufferError, "Output buffer isn't large enough"
+      when RBzip2::FFI::BZ_DATA_ERROR, RBzip2::FFI::BZ_DATA_ERROR_MAGIC,
+           RBzip2::FFI::BZ_UNEXPECTED_EOF
+        raise RBzip2::FFI::CorruptError, 'Compressed data appears to be corrupt or unreadable'
+      when RBzip2::FFI::BZ_CONFIG_ERROR
+        raise RBzip2::FFI::ConfigError, 'libbz2 has been mis-compiled'
+      else
+        raise RBzip2::FFI::Error, "Unhandled error code: #{ret}"
+    end
+  end
+
+  def read_file(length)
+    error = ::FFI::MemoryPointer.new :uint32
+    dst_buf = ::FFI::MemoryPointer.new :char, length
+
+    open_file if @bz_file.nil?
+
+    BZ2_bzRead error, @bz_file, dst_buf, length
+
+    dst_buf.read_bytes length
+  end
+
+  def initialize(io)
+    @io = io
+  end
+
+  def close
+    if @io != $stdin
+      @io = nil
+      @data = nil
+    end
+
+    close_file unless @bz_file.nil?
+  end
+
+  def close_file
+    error = ::FFI::MemoryPointer.new :uint32
+    BZ2_bzReadClose error, @bz_file
+    fclose @file
+  end
+
+  def getc
+    read 1
+  end
+
+  def gets
+    line = ''
+    loop do
+      char = getc
+      line += char
+      break if char == "\n"
+    end
+    line
+  end
+
+  def open_file(verbosity = 0, small = 0)
+    raise 'IO not a file' unless @io.is_a? File
+
+    small = 0 if small < 0
+    verbosity = 0 if verbosity < 0
+    verbosity = 4 if verbosity > 4
+
+    error = ::FFI::MemoryPointer.new :uint32
+
+    @file = fopen @io.path, 'r'
+    @bz_file = BZ2_bzReadOpen error, @file, verbosity, small, nil, 0
+  end
+
+  def read(length = nil)
+    raise 'stream closed' if @io.nil?
+
+    if length.nil?
+      factor = 4
+      compressed_data = @io.read
+      data = nil
+      while data.nil?
+        begin
+          data = self.class.decompress compressed_data, factor
+        rescue RBzip2::FFI::BufferError
+          factor = factor ** 2
+        end
+      end
+    else
+      if @io.is_a? File
+        data = read_file length
+      else
+        raise NotImplementedError
+      end
+    end
+
+    data
+  end
+
+  def size
+    if @io.is_a? StringIO
+      @io.size
+    elsif @io.is_a? File
+      @io.stat.size
+    end
+  end
+
+  def uncompressed
+    @data = read
+    @data.size
+  end
+
+  def inspect
+    "#<#{self.class}: @io=#{@io.inspect} size=#{size} uncompressed=#{uncompressed}>"
+  end
+
+end