ruby-zstds 1.1.0 → 1.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f860451f1cf45988053d7d5512e26e18a5c5b446f9f75fbf06f3a9ef3103c806
-  data.tar.gz: a224e7b3e16b3f2e1d2785fc024241813e3fb934351e5a851744eb1e2ae82783
+  metadata.gz: 882ef3555df6569fdb51a8689d2e7b8e710255f98a08a631e023ae0160d3ad86
+  data.tar.gz: 233e455b9819cab316812fd7c977735972ad5ff2eeee1a816c329e7a5b7661f1
 SHA512:
-  metadata.gz: 76ee0c342ebb71bf4ae6ac0a6139d3a8d7966da4843eece7c819b84e531fab6e501fa7ebafaa1c864f1200771b2b119ef885a827ba50da27b6112803a2858c7a
-  data.tar.gz: 575a15688165bccf52dc6f73a26fa1ac0f3a1ba0bf010384e9e34ecda8183bb904efb369635f75052420f86c84f59b906d7e46a65f301ab19e2c888d9ca96983
+  metadata.gz: 69ce6c1b92687ca88e87619a3314acf3b8c0584c56a1694730e5e5bf7139d5db3c7c280f01df79e1fb010a3c61c0c46af76a82bb1f4815b481421fb7b885f3d4
+  data.tar.gz: b77df0f6f9b98523ca3e611bbab7d5655a94c7e637fc61511096da614cb95672fab113894de375f6f406015aebc6eded0d0dabb5025644891775bec8a04cadda

data/README.md

@@ -1,14 +1,14 @@
 # Ruby bindings for zstd library
 
-| Travis | AppVeyor | Circle | Codecov | Gem   |
-| :---:  | :---:    | :---:  | :---:   | :---: |
-| [![Travis test status](https://travis-ci.com/andrew-aladev/ruby-zstds.svg?branch=master)](https://travis-ci.com/andrew-aladev/ruby-zstds) | [![AppVeyor test status](https://ci.appveyor.com/api/projects/status/github/andrew-aladev/ruby-zstds?branch=master&svg=true)](https://ci.appveyor.com/project/andrew-aladev/ruby-zstds/branch/master) | [![Circle test status](https://circleci.com/gh/andrew-aladev/ruby-zstds/tree/master.svg?style=shield)](https://circleci.com/gh/andrew-aladev/ruby-zstds/tree/master) | [![Codecov](https://codecov.io/gh/andrew-aladev/ruby-zstds/branch/master/graph/badge.svg)](https://codecov.io/gh/andrew-aladev/ruby-zstds) | [![Gem](https://img.shields.io/gem/v/ruby-zstds.svg)](https://rubygems.org/gems/ruby-zstds) |
+| AppVeyor | Circle | Github actions | Codecov | Gem  |
+| :------: | :----: | :------------: | :-----: | :--: |
+| [![AppVeyor test status](https://ci.appveyor.com/api/projects/status/github/andrew-aladev/ruby-zstds?branch=master&svg=true)](https://ci.appveyor.com/project/andrew-aladev/ruby-zstds/branch/master) | [![Circle test status](https://circleci.com/gh/andrew-aladev/ruby-zstds/tree/master.svg?style=shield)](https://circleci.com/gh/andrew-aladev/ruby-zstds/tree/master) | [![Github Actions test status](https://github.com/andrew-aladev/ruby-zstds/workflows/test/badge.svg?branch=master)](https://github.com/andrew-aladev/ruby-zstds/actions) | [![Codecov](https://codecov.io/gh/andrew-aladev/ruby-zstds/branch/master/graph/badge.svg)](https://codecov.io/gh/andrew-aladev/ruby-zstds) | [![Gem](https://img.shields.io/gem/v/ruby-zstds.svg)](https://rubygems.org/gems/ruby-zstds) |
 
 See [zstd library](https://github.com/facebook/zstd).
 
 ## Installation
 
-Please install zstd library first, use latest 1.4.3+ version.
+Please install zstd library first, use latest 1.4.0+ version.
 
 ```sh
 gem install ruby-zstds
@@ -70,16 +70,16 @@ require "zstds"
 samples = (Array.new(8) { ::SecureRandom.random_bytes(1 << 8) } + ["sample string"]).shuffle
 
 dictionary = ZSTDS::Dictionary.train samples
-File.write "dictionary.bin", dictionary.buffer
+File.write "dictionary.bin", dictionary.buffer, :mode => "wb"
 
-dictionary_buffer = File.read "dictionary.bin"
+dictionary_buffer = File.read "dictionary.bin", :mode => "rb"
 dictionary        = ZSTDS::Dictionary.new dictionary_buffer
 
 data = ZSTDS::String.compress "sample string", :dictionary => dictionary
 puts ZSTDS::String.decompress(data, :dictionary => dictionary)
 ```
 
-You can create and read `tar.zst` archives with [minitar](https://github.com/halostatue/minitar) for example.
+You can create and read `tar.zst` archives with [minitar](https://github.com/halostatue/minitar).
 
 ```ruby
 require "zstds"
@@ -113,6 +113,18 @@ get "/" do
 end
 ```
 
+All functionality (including streaming) can be used inside multiple threads with [parallel](https://github.com/grosser/parallel).
+This code will provide heavy load for your CPU.
+
+```ruby
+require "zstds"
+require "parallel"
+
+Parallel.each large_datas do |large_data|
+  ZSTDS::String.compress large_data
+end
+```
+
 ## Options
 
 | Option                          | Values         | Default    | Description |
@@ -244,7 +256,7 @@ File maintains both source and destination buffers, it accepts both `source_buff
 
 ## Stream::Writer
 
-Its behaviour is similar to builtin [`Zlib::GzipWriter`](https://ruby-doc.org/stdlib-2.7.0/libdoc/zlib/rdoc/Zlib/GzipWriter.html).
+Its behaviour is similar to builtin [`Zlib::GzipWriter`](https://ruby-doc.org/stdlib/libdoc/zlib/rdoc/Zlib/GzipWriter.html).
 
 Writer maintains destination buffer only, so it accepts `destination_buffer_length` option only.
 
@@ -282,7 +294,7 @@ Set another encodings, `nil` is just for compatibility with `IO`.
 #tell
 ```
 
-See [`IO`](https://ruby-doc.org/core-2.7.0/IO.html) docs.
+See [`IO`](https://ruby-doc.org/core/IO.html) docs.
 
 ```
 #write(*objects)
@@ -292,7 +304,7 @@ See [`IO`](https://ruby-doc.org/core-2.7.0/IO.html) docs.
 #closed?
 ```
 
-See [`Zlib::GzipWriter`](https://ruby-doc.org/stdlib-2.7.0/libdoc/zlib/rdoc/Zlib/GzipWriter.html) docs.
+See [`Zlib::GzipWriter`](https://ruby-doc.org/stdlib/libdoc/zlib/rdoc/Zlib/GzipWriter.html) docs.
 
 ```
 #write_nonblock(object, *options)
@@ -306,6 +318,9 @@ Special asynchronous methods missing in `Zlib::GzipWriter`.
 So it is possible to have asynchronous variants for these synchronous methods.
 Behaviour is the same as `IO#write_nonblock` method.
 
+All nonblock operations for file will raise `EBADF` error on Windows.
+Setting file into nonblocking mode is [not available on Windows](https://github.com/ruby/ruby/blob/master/win32/win32.c#L4388).
+
 ```
 #<<(object)
 #print(*objects)
@@ -314,11 +329,11 @@ Behaviour is the same as `IO#write_nonblock` method.
 #puts(*objects)
 ```
 
-Typical helpers, see [`Zlib::GzipWriter`](https://ruby-doc.org/stdlib-2.7.0/libdoc/zlib/rdoc/Zlib/GzipWriter.html) docs.
+Typical helpers, see [`Zlib::GzipWriter`](https://ruby-doc.org/stdlib/libdoc/zlib/rdoc/Zlib/GzipWriter.html) docs.
 
 ## Stream::Reader
 
-Its behaviour is similar to builtin [`Zlib::GzipReader`](https://ruby-doc.org/stdlib-2.7.0/libdoc/zlib/rdoc/Zlib/GzipReader.html).
+Its behaviour is similar to builtin [`Zlib::GzipReader`](https://ruby-doc.org/stdlib/libdoc/zlib/rdoc/Zlib/GzipReader.html).
 
 Reader maintains both source and destination buffers, it accepts both `source_buffer_length` and `destination_buffer_length` options.
 
@@ -353,7 +368,7 @@ Set another encodings.
 #tell
 ```
 
-See [`IO`](https://ruby-doc.org/core-2.7.0/IO.html) docs.
+See [`IO`](https://ruby-doc.org/core/IO.html) docs.
 
 ```
 #read(bytes_to_read = nil, out_buffer = nil)
@@ -363,14 +378,14 @@ See [`IO`](https://ruby-doc.org/core-2.7.0/IO.html) docs.
 #closed?
 ```
 
-See [`Zlib::GzipReader`](https://ruby-doc.org/stdlib-2.7.0/libdoc/zlib/rdoc/Zlib/GzipReader.html) docs.
+See [`Zlib::GzipReader`](https://ruby-doc.org/stdlib/libdoc/zlib/rdoc/Zlib/GzipReader.html) docs.
 
 ```
 #readpartial(bytes_to_read = nil, out_buffer = nil)
 #read_nonblock(bytes_to_read, out_buffer = nil, *options)
 ```
 
-See [`IO`](https://ruby-doc.org/core-2.7.0/IO.html) docs.
+See [`IO`](https://ruby-doc.org/core/IO.html) docs.
 
 ```
 #getbyte
@@ -393,7 +408,7 @@ See [`IO`](https://ruby-doc.org/core-2.7.0/IO.html) docs.
 #ungetline(line)
 ```
 
-Typical helpers, see [`Zlib::GzipReader`](https://ruby-doc.org/stdlib-2.7.0/libdoc/zlib/rdoc/Zlib/GzipReader.html) docs.
+Typical helpers, see [`Zlib::GzipReader`](https://ruby-doc.org/stdlib/libdoc/zlib/rdoc/Zlib/GzipReader.html) docs.
 
 ## Dictionary
 
@@ -431,11 +446,17 @@ Read dictionary id from buffer.
 Please be careful: bindings are not thread safe.
 You should lock all shared data between threads.
 
+For example: you should not use same compressor/decompressor inside multiple threads.
+Please verify that you are using each processor inside single thread at the same time.
+
+## Operating systems
+
+GNU/Linux, FreeBSD, OSX, Windows (MinGW).
+
 ## CI
 
 Please visit [scripts/test-images](scripts/test-images).
 See universal test script [scripts/ci_test.sh](scripts/ci_test.sh) for CI.
-You can run this script using many native and cross images.
 
 ## License
 

data/ext/extconf.rb

@@ -87,7 +87,7 @@ $libs = $libs.split(%r{\s})
   .uniq
   .join " "
 
-if ENV["CI"] || ENV["COVERAGE"]
+if ENV["CI"]
   $CFLAGS << " --coverage"
   $LDFLAGS << " --coverage"
 end

data/ext/zstds_ext/error.c

@@ -40,7 +40,7 @@ zstds_ext_result_t zstds_ext_get_error(ZSTD_ErrorCode error_code)
   }
 }
 
-static inline NORETURN(void raise(const char* name, const char* description))
+static inline NORETURN(void raise_error(const char* name, const char* description))
 {
   VALUE module = rb_define_module(ZSTDS_EXT_MODULE_NAME);
   VALUE error  = rb_const_get(module, rb_intern(name));
@@ -51,30 +51,30 @@ void zstds_ext_raise_error(zstds_ext_result_t ext_result)
 {
   switch (ext_result) {
     case ZSTDS_EXT_ERROR_ALLOCATE_FAILED:
-      raise("AllocateError", "allocate error");
+      raise_error("AllocateError", "allocate error");
     case ZSTDS_EXT_ERROR_VALIDATE_FAILED:
-      raise("ValidateError", "validate error");
+      raise_error("ValidateError", "validate error");
 
     case ZSTDS_EXT_ERROR_USED_AFTER_CLOSE:
-      raise("UsedAfterCloseError", "used after closed");
+      raise_error("UsedAfterCloseError", "used after closed");
     case ZSTDS_EXT_ERROR_NOT_ENOUGH_SOURCE_BUFFER:
-      raise("NotEnoughSourceBufferError", "not enough source buffer");
+      raise_error("NotEnoughSourceBufferError", "not enough source buffer");
     case ZSTDS_EXT_ERROR_NOT_ENOUGH_DESTINATION_BUFFER:
-      raise("NotEnoughDestinationBufferError", "not enough destination buffer");
+      raise_error("NotEnoughDestinationBufferError", "not enough destination buffer");
     case ZSTDS_EXT_ERROR_DECOMPRESSOR_CORRUPTED_SOURCE:
-      raise("DecompressorCorruptedSourceError", "decompressor received corrupted source");
+      raise_error("DecompressorCorruptedSourceError", "decompressor received corrupted source");
     case ZSTDS_EXT_ERROR_CORRUPTED_DICTIONARY:
-      raise("CorruptedDictionaryError", "corrupted dictionary");
+      raise_error("CorruptedDictionaryError", "corrupted dictionary");
 
     case ZSTDS_EXT_ERROR_ACCESS_IO:
-      raise("AccessIOError", "failed to access IO");
+      raise_error("AccessIOError", "failed to access IO");
     case ZSTDS_EXT_ERROR_READ_IO:
-      raise("ReadIOError", "failed to read IO");
+      raise_error("ReadIOError", "failed to read IO");
     case ZSTDS_EXT_ERROR_WRITE_IO:
-      raise("WriteIOError", "failed to write IO");
+      raise_error("WriteIOError", "failed to write IO");
 
     default:
       // ZSTDS_EXT_ERROR_UNEXPECTED
-      raise("UnexpectedError", "unexpected error");
+      raise_error("UnexpectedError", "unexpected error");
   }
 }

data/lib/zstds/file.rb

@@ -19,7 +19,7 @@ module ZSTDS
 
       options[:pledged_size] = ::File.size source
 
-      open_files(source, destination) do |source_io, destination_io|
+      open_files source, destination do |source_io, destination_io|
         ZSTDS._native_compress_io source_io, destination_io, options
       end
 
@@ -32,7 +32,7 @@ module ZSTDS
 
       options = Option.get_decompressor_options options, BUFFER_LENGTH_NAMES
 
-      open_files(source, destination) do |source_io, destination_io|
+      open_files source, destination do |source_io, destination_io|
         ZSTDS._native_decompress_io source_io, destination_io, options
       end
 

data/lib/zstds/option.rb

@@ -46,8 +46,11 @@ module ZSTDS
     def self.get_compressor_options(options, buffer_length_names)
       Validation.validate_hash options
 
-      buffer_length_defaults = buffer_length_names.each_with_object({}) { |name, defaults| defaults[name] = DEFAULT_BUFFER_LENGTH }
-      options                = COMPRESSOR_DEFAULTS.merge(buffer_length_defaults).merge options
+      buffer_length_defaults = buffer_length_names.each_with_object({}) do |name, defaults|
+        defaults[name] = DEFAULT_BUFFER_LENGTH
+      end
+
+      options = COMPRESSOR_DEFAULTS.merge(buffer_length_defaults).merge options
 
       buffer_length_names.each { |name| Validation.validate_not_negative_integer options[name] }
 
@@ -179,8 +182,11 @@ module ZSTDS
     def self.get_decompressor_options(options, buffer_length_names)
       Validation.validate_hash options
 
-      buffer_length_defaults = buffer_length_names.each_with_object({}) { |name, defaults| defaults[name] = DEFAULT_BUFFER_LENGTH }
-      options                = DECOMPRESSOR_DEFAULTS.merge(buffer_length_defaults).merge options
+      buffer_length_defaults = buffer_length_names.each_with_object({}) do |name, defaults|
+        defaults[name] = DEFAULT_BUFFER_LENGTH
+      end
+
+      options = DECOMPRESSOR_DEFAULTS.merge(buffer_length_defaults).merge options
 
       buffer_length_names.each { |name| Validation.validate_not_negative_integer options[name] }
 

data/lib/zstds/stream/abstract.rb

@@ -12,7 +12,8 @@ module ZSTDS
       # Native stream is not seekable by design.
       # Related methods like "seek" and "pos=" can't be implemented.
 
-      # It is not possible to maintain correspondance between bytes consumed from source and bytes written to destination by design.
+      # It is not possible to maintain correspondance between bytes
+      # consumed from source and bytes written to destination by design.
       # We will consume all source bytes and maintain buffer with remaining destination data.
 
       include Delegates
@@ -89,8 +90,9 @@ module ZSTDS
         end
 
         internal_encoding = args[1]
-        Validation.validate_string internal_encoding \
-          unless internal_encoding.nil? || internal_encoding.is_a?(::Encoding)
+        unless internal_encoding.nil? || internal_encoding.is_a?(::Encoding)
+          Validation.validate_string internal_encoding
+        end
 
         transcode_options = args[2]
         Validation.validate_hash transcode_options unless transcode_options.nil?

data/lib/zstds/stream/reader.rb

@@ -33,8 +33,9 @@ module ZSTDS
         source_buffer_length = @options[:source_buffer_length]
         Validation.validate_not_negative_integer source_buffer_length unless source_buffer_length.nil?
 
-        source_buffer_length = Buffer::DEFAULT_SOURCE_BUFFER_LENGTH_FOR_DECOMPRESSOR \
-          if source_buffer_length.nil? || source_buffer_length.zero?
+        if source_buffer_length.nil? || source_buffer_length.zero?
+          source_buffer_length = Buffer::DEFAULT_SOURCE_BUFFER_LENGTH_FOR_DECOMPRESSOR
+        end
 
         @source_buffer_length = source_buffer_length
       end

data/lib/zstds/stream/reader_helpers.rb

@@ -143,7 +143,7 @@ module ZSTDS
       end
 
       protected def each_string(each_proc, &block)
-        return enum_for __method__ unless block.is_a? ::Proc
+        return enum_for __method__, each_proc unless block.is_a? ::Proc
 
         loop do
           string = each_proc.call

data/lib/zstds/validation.rb

@@ -55,7 +55,9 @@ module ZSTDS
     end
 
     def self.validate_proc(value)
-      raise ValidateError, "invalid proc" unless value.is_a?(::Proc) || value.is_a?(::Method) || value.is_a?(::UnboundMethod)
+      unless value.is_a?(::Proc) || value.is_a?(::Method) || value.is_a?(::UnboundMethod)
+        raise ValidateError, "invalid proc"
+      end
     end
   end
 end

data/lib/zstds/version.rb

@@ -2,5 +2,5 @@
 # Copyright (c) 2019 AUTHORS, MIT License.
 
 module ZSTDS
-  VERSION = "1.1.0".freeze
+  VERSION = "1.1.1".freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ruby-zstds
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.1.1
 platform: ruby
 authors:
 - Andrew Aladjev
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-10-28 00:00:00.000000000 Z
+date: 2021-06-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: codecov
@@ -24,6 +24,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: json
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: minitar
   requirement: !ruby/object:Gem::Requirement
@@ -58,14 +72,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.1'
+        version: '1.3'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.1'
+        version: '1.3'
 - !ruby/object:Gem::Dependency
   name: parallel
   requirement: !ruby/object:Gem::Requirement
@@ -114,28 +128,56 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.93'
+        version: '1.16'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.93'
+        version: '1.16'
+- !ruby/object:Gem::Dependency
+  name: rubocop-minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.12'
 - !ruby/object:Gem::Dependency
   name: rubocop-performance
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.8'
+        version: '1.11'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.11'
+- !ruby/object:Gem::Dependency
+  name: rubocop-rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.8'
+        version: '0.5'
 - !ruby/object:Gem::Dependency
   name: simplecov
   requirement: !ruby/object:Gem::Requirement
@@ -150,7 +192,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description: 
+description:
 email: aladjev.andrew@gmail.com
 executables: []
 extensions:
@@ -203,7 +245,7 @@ homepage: https://github.com/andrew-aladev/ruby-zstds
 licenses:
 - MIT
 metadata: {}
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -218,8 +260,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.1.4
-signing_key: 
+rubygems_version: 3.2.15
+signing_key:
 specification_version: 4
 summary: Ruby bindings for zstd library.
 test_files: []