mime-types 2.99.3 → 3.4.1

data/README.rdoc

@@ -1,11 +1,12 @@
-= mime-types
+= mime-types for Ruby
 
 home :: https://github.com/mime-types/ruby-mime-types/
 code :: https://github.com/mime-types/ruby-mime-types/
 bugs :: https://github.com/mime-types/ruby-mime-types/issues
 rdoc :: http://rdoc.info/gems/mime-types/
-continuous integration :: {<img src="https://travis-ci.org/mime-types/ruby-mime-types.png" />}[https://travis-ci.org/mime-types/ruby-mime-types]
-test coverage :: {<img src="https://coveralls.io/repos/mime-types/ruby-mime-types/badge.png" alt="Coverage Status" />}[https://coveralls.io/r/mime-types/ruby-mime-types]
+clog :: https://github.com/mime-types/ruby-mime-types/blob/master/History.md
+continuous integration :: {<img src="https://travis-ci.org/mime-types/ruby-mime-types.svg?branch=master" alt="Build Status" />}[https://travis-ci.org/mime-types/ruby-mime-types]
+test coverage :: {<img src="https://coveralls.io/repos/mime-types/ruby-mime-types/badge.svg?branch=master&service=github" alt="Coverage Status" />}[https://coveralls.io/github/mime-types/ruby-mime-types?branch=master]
 
 == Description
 
@@ -14,19 +15,13 @@ MIME content type definitions. It can be used to determine defined filename
 extensions for MIME types, or to use filename extensions to look up the likely
 MIME type definitions.
 
-This is release 2.99.1, the first scheduled data update for mime-types 2.x. As
-of mime-types 2.99. deprecation warnings are noisy and data that has been
-deprecated is now no longer available. The data is both dropped from the data
-files and is stubbed out as empty or +nil+ values as appropriate.
-
-mime-types-2.6 was the last version of mime-types 2.x with newly available
-features, and mime-types 2.99 will only receive quarterly updates to the IANA
-registered MIME media types plus any security updates that may be required.
-
-If the loss of the deprecated data matters, be sure to set your dependency
-appropriately:
-
-   gem 'mime-types', '~> 2.6, < 2.99'
+Version 3.0 is a major release that requires Ruby 2.0 compatibility and removes
+deprecated functions. The columnar registry format introduced in 2.6 has been
+made the primary format; the registry data has been extracted from this library
+and put into {mime-types-data}[https://github.com/mime-types/mime-types-data].
+Additionally, mime-types is now licensed exclusively under the MIT licence and
+there is a code of conduct in effect. There are a number of other smaller
+changes described in the History file.
 
 === About MIME Media Types
 
@@ -41,17 +36,6 @@ recommendations, the {IANA Media Types
 registry}[https://www.iana.org/assignments/media-types/media-types.xhtml], and
 user contributions. It conforms to RFCs 2045 and 2231.
 
-=== mime-types 1.x End of Life
-
-mime-types 1.x has reached end of life and is no longer supported as of
-2015-10-27 and will receive no updates at all.
-
-=== mime-types 2.x End of Life
-
-mime-types 2.x is supported as 2.99.x receiving quarterly updates of the IANA
-registry and security updates for two years. It will reach full end of life on
-2017-11-21.
-
 === mime-types 3.x
 
 Users are encouraged to upgrade to mime-types 3.x as soon as is practical.
@@ -81,6 +65,7 @@ files). A MIME::Type stores the known information about one MIME type.
    puts text.ascii?                # => true
    puts text.obsolete?             # => false
    puts text.registered?           # => true
+   puts text.provisional?          # => false
    puts text.complete?             # => true
 
    puts text                       # => 'text/plain'
@@ -90,8 +75,8 @@ files). A MIME::Type stores the known information about one MIME type.
    puts text == 'text/x-plain'     # => false
    puts 'text/x-plain' == text     # => false
 
-   puts MIME::Type.simplified('x-appl/x-zip') # => 'appl/zip'
-   puts MIME::Type.i18n_key('x-appl/x-zip') # => 'appl.zip'
+   puts MIME::Type.simplified('x-appl/x-zip') # => 'x-appl/x-zip'
+   puts MIME::Type.i18n_key('x-appl/x-zip') # => 'x-appl.x-zip'
 
    puts text.like?('text/x-plain') # => true
    puts text.like?(MIME::Type.new('x-text/x-plain')) # => true
@@ -118,54 +103,61 @@ files). A MIME::Type stores the known information about one MIME type.
 
    xwingz = MIME::Types['application/x-Wingz'].first # => application/x-Wingz
    puts xwingz.content_type        # => 'application/x-Wingz'
-   puts xwingz.simplified          # => 'application/wingz'
+   puts xwingz.simplified          # => 'application/x-wingz'
 
 === Columnar Store
 
-mime-types 2.6 introduced an experimental columnar storage format that reduces
-the default memory footprint. It does this by selectively loading data. When a
-registry is first loaded from a columnar store, only the canonical MIME type
-and registered extensions will be loaded and the MIME type will be connected to
-its registry. When extended data is required (including +registered+,
-+obsolete+, and +use_instead+), that data is loaded from its own column file
-for all types in the registry. This load is done with a Mutex to ensure that
-the types are updated safely in a multithreaded environment.
+mime-types uses as its primary registry storage format a columnar storage
+format reducing the default memory footprint. This is done by selectively
+loading the data on a per-attribute basis. When the registry is first loaded
+from the columnar store, only the canonical MIME content type and known
+extensions and the MIME type will be connected to its loading registry. When
+other data about the type is required (including +preferred_extension+,
+<tt>obsolete?</tt>, and <tt>registered?</tt>) that data is loaded from its own
+column file for all types in the registry.
 
-Columnar storage is slated to become the default storage format for mime-types
-3.0, but until that is released, the default is still to use the JSON storage
-format. As such, columnar storage can only currently be loaded at an
-application level with the following specification in the application Gemfile:
+The load of any column data is performed with a Mutex to ensure that types are
+updated safely in a multithreaded environment. Benchmarks show that while
+columnar data loading is slower than the JSON store, it cuts the memory use by
+a third over the JSON store.
 
-   gem 'mime-types', require: 'mime/types/columnar'
+If you prefer to load all the data at once, this can be specified in your
+application Gemfile as:
 
-Projects that do not use Bundler, and libraries that wish to suggest this
-behaviour to applications are encouraged to require this directly, but only if
-you specify a dependency on mime-types 2.6.
+   gem 'mime-types', require: 'mime/types/full'
 
-   require 'mime/types/columnar'
+Projects that do not use Bundler should +require+ the same:
+
+   require 'mime/types/full'
 
-Although this require will not be necessary after mime-types 3, it will work
-through at least {version
+Libraries that use mime-types are discouraged from choosing the JSON store.
+
+For applications and clients that used mime-types 2.6 when the columnar store
+was introduced, the require used previously will still work through at least
+{version
 4}[https://github.com/mime-types/ruby-mime-types/pull/96#issuecomment-100725400]
-and possibly beyond.
+and possibly beyond; it is effectively an empty operation. You are recommended
+to change your Gemfile as soon as is practical.
+
+   require 'mime/types/columnar'
 
-Note that the new Columnar class (MIME::Type::Columnar) and module
-(MIME::Types::Columnar) are considered private variant implementations of
-MIME::Type and MIME::Types and the specific implementation should not be relied
-upon by consumers of the mime-types library. Instead, depend on the public
-implementations only.
+Note that MIME::Type::Columnar and MIME::Types::Columnar are considered private
+variant implementations of MIME::Type and MIME::Types and the specific
+implementation should not be relied upon by consumers of the mime-types
+library. Instead, depend on the public implementations (MIME::Type and
+MIME::Types) only.
 
 === Cached Storage
 
-Since version 2.0, mime-types has supported a cache of MIME types based on
-<tt>Marshal.dump</tt>. The cache is invalidated for each released version of
-mime-types so that version 2.5 is not reused for version 2.6. If the
+mime-types supports a cache of MIME types using <tt>Marshal.dump</tt>. The
+cache is invalidated for each version of the mime-types-data gem so that data
+version 3.2015.1201 will not be reused with data version 3.2016.0101. If the
 environment variable +RUBY_MIME_TYPES_CACHE+ is set to a cache file, mime-types
 will attempt to load the MIME type registry from the cache file. If it cannot,
 it will load the types normally and then saves the registry to the cache file.
 
-The current mime-types cache is not compatible with the columnar storage
-format. This will be resolved for mime-types 3.
+The caching works with both full stores and columnar stores. Only the data that
+has been loaded prior to saving the cache will be stored.
 
 == mime-types Modified Semantic Versioning
 
@@ -183,22 +175,20 @@ MAJOR.MINOR:
    incremented, MINOR will be set to zero, and PATCH will be reset to the
    implied zero.
 
-2. If an API (code) feature is added that does not break compatibilty OR if
-   there are MIME types added, removed, or changed in the registry, the MINOR
-   version will be incremented and PATCH will be reset to the implied zero.
+2. If an API (code) feature is added that does not break compatibility, the
+   MINOR version will be incremented and PATCH will be reset to the implied zero.
 
 3. If there is a bugfix to a feature added in the most recent MAJOR.MINOR
-   release, OR if purely typographical errors are fixed in MIME types, the
-   implied PATCH value will be incremented resulting in MAJOR.MINOR.PATCH.
+   release, the implied PATCH value will be incremented resulting in
+   MAJOR.MINOR.PATCH.
 
-In practical terms, there should be a MINOR release roughly monthly to track
-updated or changed MIME types from the official IANA registry. This does not
-indicate when new API features have been added, but all minor versions of
-mime-types 2.x will be backwards compatible; the interfaces marked deprecated
-will be removed in mime-types 3.x.
+In practical terms, there will be fewer releases of mime-types focussing on
+features because of the existence of the [mime-types-data][] gem, and if
+features are marked deprecated in the course of mime-types 3.x, they will not
+be removed until mime-types 4.x or possibly later.
 
-:include: Code-of-Conduct.rdoc
+{Code of Conduct}[Code-of-Conduct_md.html]
 
-:include: Contributing.rdoc
+{Contributing}[Contributing_md.html]
 
-:include: Licence.rdoc
+{Licence}[Licence_md.html]

data/Rakefile

@@ -1,77 +1,108 @@
 # frozen_string_literal: true
 
-require 'rubygems'
-require 'hoe'
-require 'rake/clean'
+require "rubygems"
+require "hoe"
+require "rake/clean"
+
+# This is required until https://github.com/seattlerb/hoe/issues/112 is fixed
+class Hoe
+  def with_config
+    config = Hoe::DEFAULT_CONFIG
+
+    rc = File.expand_path("~/.hoerc")
+    homeconfig = load_config(rc)
+    config = config.merge(homeconfig)
+
+    localconfig = load_config(File.expand_path(File.join(Dir.pwd, ".hoerc")))
+    config = config.merge(localconfig)
+
+    yield config, rc
+  end
+
+  def load_config(name)
+    File.exist?(name) ? safe_load_yaml(name) : {}
+  end
+
+  def safe_load_yaml(name)
+    return safe_load_yaml_file(name) if YAML.respond_to?(:safe_load_file)
+
+    data = IO.binread(name)
+    YAML.safe_load(data, permitted_classes: [Regexp])
+  rescue
+    YAML.safe_load(data, [Regexp])
+  end
+
+  def safe_load_yaml_file(name)
+    YAML.safe_load_file(name, permitted_classes: [Regexp])
+  rescue
+    YAML.safe_load_file(name, [Regexp])
+  end
+end
 
 Hoe.plugin :doofus
 Hoe.plugin :gemspec2
 Hoe.plugin :git
 Hoe.plugin :minitest
-Hoe.plugin :travis
-Hoe.plugin :email unless ENV['CI'] or ENV['TRAVIS']
+Hoe.plugin :email unless ENV["CI"]
 
-spec = Hoe.spec 'mime-types' do
-  developer('Austin Ziegler', 'halostatue@gmail.com')
+spec = Hoe.spec "mime-types" do
+  developer("Austin Ziegler", "halostatue@gmail.com")
   self.need_tar = true
 
-  require_ruby_version '>= 1.9.2'
-
-  self.history_file = 'History.rdoc'
-  self.readme_file = 'README.rdoc'
-  self.extra_rdoc_files = FileList['*.rdoc'].to_a
-
-  self.licenses = ['MIT', 'Artistic-2.0', 'GPL-2.0']
-
-  extra_dev_deps << ['hoe-doofus', '~> 1.0']
-  extra_dev_deps << ['hoe-gemspec2', '~> 1.1']
-  extra_dev_deps << ['hoe-git', '~> 1.6']
-  extra_dev_deps << ['hoe-rubygems', '~> 1.0']
-  extra_dev_deps << ['hoe-travis', '~> 1.2']
-  extra_dev_deps << ['minitest', '~> 5.4']
-  extra_dev_deps << ['minitest-autotest', '~> 1.0']
-  extra_dev_deps << ['minitest-focus', '~> 1.0']
-  extra_dev_deps << ['rake', '~> 10.0']
-  extra_dev_deps << ['fivemat', '~> 1.3' ]
-  extra_dev_deps << ['minitest-rg', '~> 5.2']
-end
+  require_ruby_version ">= 2.0"
 
-task :support do
-  %w(lib support).each { |path|
-    $LOAD_PATH.unshift(File.join(Rake.application.original_dir, path))
-  }
-end
+  self.history_file = "History.md"
+  self.readme_file = "README.rdoc"
 
-task 'support:nokogiri' => :support do
-  begin
-    gem 'nokogiri'
-  rescue Gem::LoadError
-    raise 'Nokogiri is not installed. Please install it.'
+  license "MIT"
+
+  extra_deps << ["mime-types-data", "~> 3.2015"]
+
+  extra_dev_deps << ["hoe-doofus", "~> 1.0"]
+  extra_dev_deps << ["hoe-gemspec2", "~> 1.1"]
+  extra_dev_deps << ["hoe-git", "~> 1.6"]
+  extra_dev_deps << ["hoe-rubygems", "~> 1.0"]
+  extra_dev_deps << ["standard", "~> 1.0"]
+  extra_dev_deps << ["minitest", "~> 5.4"]
+  extra_dev_deps << ["minitest-autotest", "~> 1.0"]
+  extra_dev_deps << ["minitest-focus", "~> 1.0"]
+  extra_dev_deps << ["minitest-bonus-assertions", "~> 3.0"]
+  extra_dev_deps << ["minitest-hooks", "~> 1.4"]
+  extra_dev_deps << ["rake", ">= 10.0", "< 14.0"]
+
+  if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("2.0")
+    extra_dev_deps << ["simplecov", "~> 0.7"]
   end
 end
 
 namespace :benchmark do
-  desc 'Benchmark Load Times'
-  task :load, [ :repeats ] => :support do |_, args|
-    require 'benchmarks/load'
+  task :support do
+    %w[lib support].each { |path|
+      $LOAD_PATH.unshift(File.join(Rake.application.original_dir, path))
+    }
+  end
+
+  desc "Benchmark Load Times"
+  task :load, [:repeats] => "benchmark:support" do |_, args|
+    require "benchmarks/load"
     Benchmarks::Load.report(
-      File.join(Rake.application.original_dir, 'lib'),
+      File.join(Rake.application.original_dir, "lib"),
       args.repeats
     )
   end
 
-  desc 'Allocation counts'
-  task :allocations, [ :top_x, :mime_types_only ] => :support do |_, args|
-    require 'benchmarks/load_allocations'
+  desc "Allocation counts"
+  task :allocations, [:top_x, :mime_types_only] => "benchmark:support" do |_, args|
+    require "benchmarks/load_allocations"
     Benchmarks::LoadAllocations.report(
       top_x: args.top_x,
       mime_types_only: args.mime_types_only
     )
   end
 
-  desc 'Columnar allocation counts'
-  task 'allocations:columnar', [ :top_x, :mime_types_only ] => :support do |_, args|
-    require 'benchmarks/load_allocations'
+  desc "Columnar allocation counts"
+  task "allocations:columnar", [:top_x, :mime_types_only] => "benchmark:support" do |_, args|
+    require "benchmarks/load_allocations"
     Benchmarks::LoadAllocations.report(
       columnar: true,
       top_x: args.top_x,
@@ -79,89 +110,161 @@ namespace :benchmark do
     )
   end
 
-  desc 'Object counts'
-  task objects: :support do
-    require 'benchmarks/object_counts'
+  desc "Columnar allocation counts (full load)"
+  task "allocations:columnar:full", [:top_x, :mime_types_only] => "benchmark:support" do |_, args|
+    require "benchmarks/load_allocations"
+    Benchmarks::LoadAllocations.report(
+      columnar: true,
+      top_x: args.top_x,
+      mime_types_only: args.mime_types_only,
+      full: true
+    )
+  end
+
+  desc "Memory profiler"
+  task :memory, [:top_x, :mime_types_only] => "benchmark:support" do |_, args|
+    require "benchmarks/memory_profiler"
+    Benchmarks::ProfileMemory.report(
+      mime_types_only: args.mime_types_only,
+      top_x: args.top_x
+    )
+  end
+
+  desc "Columnar memory profiler"
+  task "memory:columnar", [:top_x, :mime_types_only] => "benchmark:support" do |_, args|
+    require "benchmarks/memory_profiler"
+    Benchmarks::ProfileMemory.report(
+      columnar: true,
+      mime_types_only: args.mime_types_only,
+      top_x: args.top_x
+    )
+  end
+
+  desc "Columnar allocation counts (full load)"
+  task "memory:columnar:full", [:top_x, :mime_types_only] => "benchmark:support" do |_, args|
+    require "benchmarks/memory_profiler"
+    Benchmarks::ProfileMemory.report(
+      columnar: true,
+      full: true,
+      top_x: args.top_x,
+      mime_types_only: args.mime_types_only
+    )
+  end
+
+  desc "Object counts"
+  task objects: "benchmark:support" do
+    require "benchmarks/object_counts"
     Benchmarks::ObjectCounts.report
   end
 
-  desc 'Columnar object counts'
-  task 'objects:columnar' => :support do
-    require 'benchmarks/object_counts'
+  desc "Columnar object counts"
+  task "objects:columnar" => "benchmark:support" do
+    require "benchmarks/object_counts"
     Benchmarks::ObjectCounts.report(columnar: true)
   end
+
+  desc "Columnar object counts (full load)"
+  task "objects:columnar:full" => "benchmark:support" do
+    require "benchmarks/object_counts"
+    Benchmarks::ObjectCounts.report(columnar: true, full: true)
+  end
 end
 
-namespace :mime do
-  desc 'Download the current MIME type registrations from IANA.'
-  task :iana, [ :destination ] => 'support:nokogiri' do |_, args|
-    require 'iana_registry'
-    IANARegistry.download(to: args.destination)
+namespace :profile do
+  directory "tmp/profile"
+
+  CLEAN.add "tmp"
+
+  def ruby_prof(script)
+    require "pathname"
+    output = Pathname("tmp/profile").join(script)
+    output.mkpath
+    script = Pathname("support/profile").join("#{script}.rb")
+
+    args = [
+      "-W0",
+      "-Ilib",
+      "-S", "ruby-prof",
+      "-R", "mime/types",
+      "-s", "self",
+      "-p", "multi",
+      "-f", output.to_s,
+      script.to_s
+    ]
+    ruby args.join(" ")
+  end
+
+  task full: "tmp/profile" do
+    ruby_prof "full"
+  end
+
+  task columnar: :support do
+    ruby_prof "columnar"
+  end
+
+  task "columnar:full" => :support do
+    ruby_prof "columnar_full"
   end
+end
 
-  desc 'Download the current MIME type configuration from Apache.'
-  task :apache, [ :destination ] => 'support:nokogiri' do |_, args|
-    require 'apache_mime_types'
-    ApacheMIMETypes.download(to: args.destination)
+if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new("2.0")
+  namespace :test do
+    desc "Run test coverage"
+    task :coverage do
+      spec.test_prelude = [
+        'require "simplecov"',
+        'SimpleCov.start("test_frameworks") { command_name "Minitest" }',
+        'gem "minitest"'
+      ].join("; ")
+      Rake::Task["test"].execute
+    end
   end
 end
 
 namespace :convert do
   namespace :docs do
     task :setup do
-      gem 'rdoc'
-      require 'rdoc/rdoc'
+      gem "rdoc"
+      require "rdoc/rdoc"
       @doc_converter ||= RDoc::Markup::ToMarkdown.new
     end
 
-    %w(README History History-Types).each do |name|
-      rdoc = "#{name}.rdoc"
-      mark = "#{name}.md"
+    FileList["*.rdoc"].each do |name|
+      rdoc = name
+      mark = "#{File.basename(name, ".rdoc")}.md"
 
-      file mark => [ rdoc, :setup ] do |t|
+      file mark => [rdoc, :setup] do |t|
         puts "#{rdoc} => #{mark}"
-        File.open(t.name, 'wb') { |target|
+        File.open(t.name, "wb") { |target|
           target.write @doc_converter.convert(IO.read(t.prerequisites.first))
         }
       end
 
       CLEAN.add mark
 
-      task run: [ mark ]
+      task run: [mark]
     end
   end
 
-  desc 'Convert documentation from RDoc to Markdown'
-  task docs: 'convert:docs:run'
-
-  namespace :yaml do
-    desc 'Convert from YAML to JSON'
-    task :json, [ :source, :destination, :multiple_files ] => :support do |_, args|
-      require 'convert'
-      Convert.from_yaml_to_json(args)
-    end
+  desc "Convert documentation from RDoc to Markdown"
+  task docs: "convert:docs:run"
+end
 
-    desc 'Convert from YAML to Columnar'
-    task :columnar, [ :source, :destination ] => :support do |_, args|
-      require 'convert/columnar'
-      Convert::Columnar.from_yaml_to_columnar(args)
-    end
+namespace :deps do
+  task :top, [:number] => "benchmark:support" do |_, args|
+    require "deps"
+    Deps.run(args)
   end
+end
 
-  namespace :json do
-    desc 'Convert from JSON to YAML'
-    task :yaml, [ :source, :destination, :multiple_files ] => :support do |_, args|
-      require 'convert'
-      Convert.from_json_to_yaml(args)
-    end
+task :console do
+  arguments = %w[irb]
+  arguments.push(*spec.spec.require_paths.map { |dir| "-I#{dir}" })
+  arguments.push("-r#{spec.spec.name.gsub("-", File::SEPARATOR)}")
+  unless system(*arguments)
+    error "Command failed: #{show_command}"
+    abort
   end
 end
 
-desc 'Default conversion from YAML to JSON and Columnar'
-task convert: [ 'convert:yaml:json', 'convert:yaml:columnar' ]
-
-Rake::Task['gem'].prerequisites.unshift('convert')
-Rake::Task['gem'].prerequisites.unshift('git:manifest')
-Rake::Task['gem'].prerequisites.unshift('gemspec')
-
 # vim: syntax=ruby