minitar 0.6.1 → 0.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 6f701b9b323414ca305fd29f288088f7b6656342
-  data.tar.gz: f0875b48ec987190fdd2a8c6e6d5785b87958af7
+SHA256:
+  metadata.gz: fe4dcb6889d7dd3e767dc3329f8e874d4359577f219b0a76df859617a253c4fb
+  data.tar.gz: 345a765b717201d0a47dbb4d52572d68d51797fb41f7df0f314e9a43b160e6de
 SHA512:
-  metadata.gz: b51ac6a3c8c9494f6d17392e6e7969836d2fae501e47f6083cbf0b3646dea93eb4838a25986b3723c071c791ac0ac770e8229276a77a5c06021398d82cbc4a0c
-  data.tar.gz: 9423c08b9f8426d415b3f77647c53cf744180dbdc328f4f40e15a9632171696810902a341fa654133f3e4c8ca6b84277dd93892426bb553ed309785f47114228
+  metadata.gz: faf3628dd94d2a9f85c86a7662b8699911d022d5b7e75ea2f41effd6fd896252721e953203796cd99739f7c9e756066250d78e613ee606271d79c36753836df2
+  data.tar.gz: b0311f7166e41805adce3709c53a0acda7dcfe5b732abb16d278706b5e54b56cb2477e3e51bc0bf8d5f6143e61421fd11860a3e1fca22c57ea405f751fa8b369

data/Contributing.md

@@ -78,6 +78,10 @@ Thanks to everyone who has contributed to minitar:
 *   Mike Furr
 *   Pete Fritchman
 *   Zach Dennis
+*   ooooooo\_q
+*   Kazuyoshi Kato
+*   dearblue
+*   Kevin McDermott
 
 [Minitest]: https://github.com/seattlerb/minitest
 [quality commit messages]: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html

data/History.md

@@ -1,3 +1,26 @@
+## 0.7 / 2018-02-19
+
+*   Fixed issue [#28][] with a modified version of PR [#29][] covering the
+    security policy and position for Minitar. Thanks so much to ooooooo\_q for
+    the report and an initial patch. Additional information was added as
+    [#30][].
+
+*   dearblue contributed PR [#33][] providing a fix for Minitar::Reader when
+    the IO-like object does not have a `#pos` method.
+
+*   Kevin McDermott contributed PR [#34][] so that an InvalidTarStream is
+    raised if the tar header is not valid, preventing incorrect streaming of
+    files from a non-tarfile. This is a minor breaking change, so the version
+    has been bumped accordingly.
+
+*   Kazuyoshi Kato contributed PR [#26][] providing support for the GNU tar
+    long filename extension.
+
+*   Addressed a potential DOS with negative size fields in tar headers
+    ([#31][]). This has been handled in two ways: the size field in a tar
+    header is interpreted as a strict octal value and the Minitar reader will
+    raise an InvalidTarStream if the size ends up being negative anyway.
+
 ## 0.6.1 / 2017-02-07
 
 *   Fixed issue [#24][] where streams were being improperly closed immediately
@@ -115,3 +138,8 @@
 [#16]: https://github.com/halostatue/minitar/issues/16
 [#23]: https://github.com/halostatue/minitar/issues/23
 [#24]: https://github.com/halostatue/minitar/issues/24
+[#26]: https://github.com/halostatue/minitar/issues/26
+[#28]: https://github.com/halostatue/minitar/issues/28
+[#29]: https://github.com/halostatue/minitar/issues/29
+[#30]: https://github.com/halostatue/minitar/issues/30
+[#33]: https://github.com/halostatue/minitar/issues/33

data/README.rdoc

@@ -14,9 +14,12 @@ coveralls :: {<img src="https://coveralls.io/repos/halostatue/minitar/badge.svg"
 The minitar library is a pure-Ruby library that provides the ability to deal
 with POSIX tar(1) archive files.
 
-This is release 0.6, providing a number of bug fixes including a directory
-traversal vulnerability, CVE-2016-10173. This release starts the migration and
-modernization of the code:
+This is release 0.7, providing fixes for several issues and clarifying the
+Minitar security stance. There are two minor breaking changes in this version
+so that exceptions will be thrown if a negative size is provided in a tar
+stream header or if the tar stream header is otherwise invalid.
+
+This release continues the migration and modernization of the code:
 
 *   the licence has been changed to match the modern Ruby licensing scheme
     (Ruby and Simplified BSD instead of Ruby and GNU GPL);
@@ -73,6 +76,34 @@ wrapped data stream object.
     tar.close
   end
 
+== Minitar and Security
+
+Minitar aims to be secure by default for the data *inside* of a tarfile. If
+there are any security issues discovered, please feel free to open an issue.
+Should you wish to make a more confidential report, you can find my PGP key
+information at {Keybase}[https://keybase.io/halostatue]. Bear with me: I do not
+use PGP regularly, so it may take some time to remember the command invocations
+required to successfully handle this.
+
+Minitar does *not* perform validation of path names provided to the convenience
+calsses Minitar::Output and Minitar::Input, which use Kernel.open for their
+underlying implementations when not given an IO-like object.
+
+Improper use of these classes with arbitrary input filenames may leave your
+your software to the same class of vulnerability as reported for Net::FTP
+({CVE-2017-17405}[https://nvd.nist.gov/vuln/detail/CVE-2017-17405]). Of
+particular note, "if the localfile argument starts with the '|' pipe character,
+the command following the pipe character is executed."
+
+Additionally, the use of the `open-uri` library (which extends Kernel.open with
+transparent implementations of Net::HTTP, Net::HTTPS, and Net::FTP), there are
+other possible vulnerabilities when accepting arbitrary input, as
+{detailed}[https://sakurity.com/blog/2015/02/28/openuri.html] by Egor Homakov.
+
+These security vulnerabilities may be avoided, even with the Minitar::Output
+and Minitar::Input convenience classes, by providing IO-like objects instead of
+pathname-like objects as the source or destination of these classes.
+
 == minitar Semantic Versioning
 
 The minitar library uses a {Semantic Versioning}[http://semver.org/] scheme

data/lib/archive/tar/minitar.rb

@@ -73,7 +73,7 @@ end
 #       tar.close
 #     end
 module Archive::Tar::Minitar
-  VERSION = '0.6.1'.freeze # :nodoc:
+  VERSION = '0.7'.freeze # :nodoc:
 
   # The base class for any minitar error.
   Error = Class.new(::StandardError)
@@ -91,6 +91,8 @@ module Archive::Tar::Minitar
   # The exception raised when a file contains a relative path in secure mode
   # (the default for this version).
   SecureRelativePathError = Class.new(Error)
+  # The exception raised when a file contains an invalid Posix header.
+  InvalidTarStream = Class.new(Error)
 
   class << self
     # Tests if +path+ refers to a directory. Fixes an apparently

data/lib/archive/tar/minitar/posix_header.rb

@@ -36,6 +36,9 @@ module Archive::Tar::Minitar; end
 # unrecognized typeflag value as a regular file."
 class Archive::Tar::Minitar::PosixHeader
   BLOCK_SIZE = 512
+  MAGIC_BYTES = 'ustar'.freeze
+
+  GNU_EXT_LONG_LINK = '././@LongLink'
 
   # Fields that must be set in a POSIX tar(1) header.
   REQUIRED_FIELDS = [ :name, :size, :prefix, :mode ].freeze
@@ -81,7 +84,7 @@ class Archive::Tar::Minitar::PosixHeader
       mode      = fields.shift.oct
       uid       = fields.shift.oct
       gid       = fields.shift.oct
-      size      = fields.shift.oct
+      size      = strict_oct(fields.shift)
       mtime     = fields.shift.oct
       checksum  = fields.shift.oct
       typeflag  = fields.shift
@@ -116,6 +119,13 @@ class Archive::Tar::Minitar::PosixHeader
         :linkname => linkname
       )
     end
+
+    private
+
+    def strict_oct(string)
+      return string.oct if string =~ /\A[0-7]*\z/
+      raise ArgumentError, "#{string.inspect} is not a valid octal string"
+    end
   end
 
   # Creates a new PosixHeader. A PosixHeader cannot be created unless
@@ -128,7 +138,7 @@ class Archive::Tar::Minitar::PosixHeader
     v[:mtime] = v[:mtime].to_i
     v[:checksum] ||= ''
     v[:typeflag] ||= '0'
-    v[:magic]    ||= 'ustar'
+    v[:magic]    ||= MAGIC_BYTES
     v[:version]  ||= '00'
 
     FIELDS.each do |f|
@@ -143,10 +153,15 @@ class Archive::Tar::Minitar::PosixHeader
     @empty
   end
 
+  # Indicates if the header has a valid magic value.
+  def valid?
+    empty? || @magic == MAGIC_BYTES
+  end
+
   # Returns +true+ if the header is a long name special header which indicates
   # that the next block of data is the filename.
   def long_name?
-    typeflag == 'L' && name == '././@LongLink'
+    typeflag == 'L' && name == GNU_EXT_LONG_LINK
   end
 
   # A string representation of the header.

data/lib/archive/tar/minitar/reader.rb

@@ -5,6 +5,7 @@ module Archive::Tar::Minitar
   # stream may be sequential or random access, but certain features only work
   # with random access data streams.
   class Reader
+    include Enumerable
     # This marks the EntryStream closed for reading without closing the
     # actual data stream.
     module InvalidEntryStream
@@ -179,7 +180,7 @@ module Archive::Tar::Minitar
     # Creates and returns a new Reader object.
     def initialize(io)
       @io = io
-      @init_pos = io.pos
+      @init_pos = io.pos rescue nil
     end
 
     # Resets the read pointer to the beginning of data stream. Do not call
@@ -207,8 +208,11 @@ module Archive::Tar::Minitar
         return if @io.eof?
 
         header = Archive::Tar::Minitar::PosixHeader.from_stream(@io)
+        raise Archive::Tar::Minitar::InvalidTarStream unless header.valid?
         return if header.empty?
 
+        raise Archive::Tar::Minitar::InvalidTarStream if header.size < 0
+
         if header.long_name?
           name = @io.read(512).rstrip
           header = PosixHeader.from_stream(@io)

data/lib/archive/tar/minitar/writer.rb

@@ -132,11 +132,8 @@ module Archive::Tar::Minitar
     #    end
     def add_file_simple(name, opts = {}) # :yields BoundedWriteStream:
       raise ClosedStream if @closed
-      name, prefix = split_name(name)
 
       header = {
-        :prefix => prefix,
-        :name   => name,
         :mode   => opts.fetch(:mode, 0o644),
         :mtime  => opts.fetch(:mtime, nil),
         :gid    => opts.fetch(:gid, nil),
@@ -161,7 +158,7 @@ module Archive::Tar::Minitar
 
       header[:size] = size
 
-      @io.write(PosixHeader.new(header))
+      write_header(name, header)
 
       os = BoundedWriteStream.new(@io, opts[:size])
       if block_given?
@@ -208,8 +205,6 @@ module Archive::Tar::Minitar
         raise Archive::Tar::Minitar::NonSeekableStream
       end
 
-      name, prefix = split_name(name)
-
       init_pos = @io.pos
       @io.write("\0" * 512) # placeholder for the header
 
@@ -222,15 +217,13 @@ module Archive::Tar::Minitar
       final_pos, @io.pos = @io.pos, init_pos
 
       header = {
-        :name => name,
         :mode => opts[:mode],
         :mtime => opts[:mtime],
         :size => size,
         :gid => opts[:gid],
         :uid => opts[:uid],
-        :prefix => prefix
       }
-      @io.write(PosixHeader.new(header))
+      write_header(name, header)
       @io.pos = final_pos
     end
 
@@ -238,18 +231,15 @@ module Archive::Tar::Minitar
     def mkdir(name, opts = {})
       raise ClosedStream if @closed
 
-      name, prefix = split_name(name)
       header = {
-        :name => name,
         :mode => opts[:mode],
         :typeflag => '5',
         :size => 0,
         :gid => opts[:gid],
         :uid => opts[:uid],
         :mtime => opts[:mtime],
-        :prefix => prefix
       }
-      @io.write(PosixHeader.new(header))
+      write_header(name, header)
       nil
     end
 
@@ -275,11 +265,27 @@ module Archive::Tar::Minitar
 
     private
 
-    def split_name(name)
-      # TODO: Enable long-filename write support.
+    def write_header(long_name, header)
+      short_name, prefix, needs_long_name = split_name(long_name)
+
+      if needs_long_name
+        long_name_header = {
+          :prefix => '',
+          :name => PosixHeader::GNU_EXT_LONG_LINK,
+          :typeflag => 'L',
+          :size => long_name.length,
+          :mode => 0,
+        }
+        @io.write(PosixHeader.new(long_name_header))
+        @io.write(long_name)
+        @io.write("\0" * (512 - (long_name.length % 512)))
+      end
 
-      raise FileNameTooLong if name.size > 256
+      new_header = header.merge({ :name => short_name, :prefix => prefix })
+      @io.write(PosixHeader.new(new_header))
+    end
 
+    def split_name(name)
       if name.size <= 100
         prefix = ''
       else
@@ -297,11 +303,9 @@ module Archive::Tar::Minitar
         prefix = (parts + [nxt]).join('/')
 
         name = newname
-
-        raise FileNameTooLong if name.size > 100 || prefix.size > 155
       end
 
-      [ name, prefix ]
+      [ name, prefix, (name.size > 100 || prefix.size > 155) ]
     end
   end
 end

data/test/test_tar_header.rb

@@ -71,4 +71,19 @@ class TestTarHeader < Minitest::Test
     h = Archive::Tar::Minitar::PosixHeader.from_stream header
     assert_equal('a ', h.name)
   end
+
+  def test_valid_with_valid_header
+    header = tar_file_header('a' * 100, '', 0o12345, 10)
+    header = StringIO.new(header)
+    h = Archive::Tar::Minitar::PosixHeader.from_stream header
+
+    assert(h.valid?)
+  end
+
+  def test_valid_with_invalid_header
+    header = StringIO.new("testing")
+    h = Archive::Tar::Minitar::PosixHeader.from_stream header
+
+    refute(h.valid?)
+  end
 end

data/test/test_tar_output.rb

@@ -4,9 +4,11 @@ require 'minitar'
 require 'minitest_helper'
 
 class TestTarOutput < Minitest::Test
+  NAMES = ['a', 'b', 'c', 'd' * 200]
+
   def setup
     FileUtils.mkdir_p('data__')
-    %w(a b c).each do |filename|
+    NAMES.each do |filename|
       name = File.join('data__', filename)
       File.open(name, 'wb') { |f|
         f.puts "#{name}: 123456789012345678901234567890"
@@ -30,7 +32,7 @@ class TestTarOutput < Minitest::Test
   def test_file_looks_good
     Minitar::Output.open(@tarfile) do |os|
       Dir.chdir('data__') do
-        %w(a b c).each do |name|
+        NAMES.each do |name|
           stat = File.stat(name)
           opts = { :size => stat.size, :mode => 0o644 }
           os.tar.add_file_simple(name, opts) do |ss|
@@ -41,19 +43,10 @@ class TestTarOutput < Minitest::Test
     end
     ff = File.open(@tarfile, 'rb')
     Minitar::Reader.open(ff) do |is|
-      ii = 0
-      is.each do |entry|
-        case ii
-        when 0
-          assert_equal('a', entry.name)
-        when 1
-          assert_equal('b', entry.name)
-        when 2
-          assert_equal('c', entry.name)
-        end
-        ii += 1
+      names_from_tar = is.map do |entry|
+        entry.name
       end
-      assert_equal(3, ii)
+      assert_equal(NAMES, names_from_tar)
     end
   ensure
     ff.close if ff

data/test/test_tar_reader.rb

@@ -158,4 +158,14 @@ class TestTarReader < Minitest::Test
       end
     end
   end
+
+  def test_read_invalid_tar_file
+    assert_raises Archive::Tar::Minitar::InvalidTarStream do
+      Minitar::Reader.open(StringIO.new("testing")) do |r|
+        r.each_entry do |entry|
+          fail "invalid tar file should not read files"
+        end
+      end
+    end
+  end
 end

data/test/test_tar_writer.rb

@@ -106,21 +106,24 @@ class TestTarWriter < Minitest::Test
         @dummyos.data[2 * i * 512, 512]
       )
     end
-    assert_raises(Minitar::FileNameTooLong) do
-      @os.add_file_simple(File.join('a' * 152, 'b' * 10, 'a' * 92),
-        :mode => 0o644, :size => 10) {}
-    end
-    assert_raises(Minitar::FileNameTooLong) do
-      @os.add_file_simple(File.join('a' * 162, 'b' * 10),
-        :mode => 0o644, :size => 10) {}
-    end
-    assert_raises(Minitar::FileNameTooLong) do
-      @os.add_file_simple(File.join('a' * 10, 'b' * 110),
-        :mode => 0o644, :size => 10) {}
-    end
+  end
+
+  def test_file_name_is_long
+    @dummyos.reset
+
+    @os.add_file_simple(File.join('a' * 152, 'b' * 10, 'c' * 92),
+                        :mode => 0o644, :size => 10) {}
+    @os.add_file_simple(File.join('d' * 162, 'e' * 10),
+                        :mode => 0o644, :size => 10) {}
+    @os.add_file_simple(File.join('f' * 10, 'g' * 110),
+                        :mode => 0o644, :size => 10) {}
     # Issue #6.
-    assert_raises(Minitar::FileNameTooLong) do
-      @os.add_file_simple('a' * 114, :mode => 0o644, :size => 10) {}
+    @os.add_file_simple('a' * 114, :mode => 0o644, :size => 10) {}
+
+    # "././@LongLink", a file name, its actual header, its data, ...
+    4.times do |i|
+      assert_equal(Minitar::PosixHeader::GNU_EXT_LONG_LINK,
+                   @dummyos.data[4 * i * 512, 32].rstrip)
     end
   end
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: minitar
 version: !ruby/object:Gem::Version
-  version: 0.6.1
+  version: '0.7'
 platform: ruby
 authors:
 - Austin Ziegler
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-02-08 00:00:00.000000000 Z
+date: 2018-10-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -16,14 +16,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '5.10'
+        version: '5.11'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '5.10'
+        version: '5.11'
 - !ruby/object:Gem::Dependency
   name: hoe-doofus
   requirement: !ruby/object:Gem::Requirement
@@ -154,21 +154,24 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.16'
+        version: '3.17'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.16'
+        version: '3.17'
 description: |-
   The minitar library is a pure-Ruby library that provides the ability to deal
   with POSIX tar(1) archive files.
 
-  This is release 0.6, providing a number of bug fixes including a directory
-  traversal vulnerability, CVE-2016-10173. This release starts the migration and
-  modernization of the code:
+  This is release 0.7, providing fixes for several issues and clarifying the
+  Minitar security stance. There are two minor breaking changes in this version
+  so that exceptions will be thrown if a negative size is provided in a tar
+  stream header or if the tar stream header is otherwise invalid.
+
+  This release continues the migration and modernization of the code:
 
   *   the licence has been changed to match the modern Ruby licensing scheme
       (Ruby and Simplified BSD instead of Ruby and GNU GPL);
@@ -248,7 +251,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.5.1
+rubygems_version: 2.7.7
 signing_key: 
 specification_version: 4
 summary: The minitar library is a pure-Ruby library that provides the ability to deal