mime-types 2.5 → 2.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 3023bc21b31749c2bff1ad6ee0292dc9c31bcf50
-  data.tar.gz: e9bd5ffbd736396869836738617dd3237b0f7e5b
+  metadata.gz: 393ac8426babd4f63f6cd7dd13ec79df643c4dcf
+  data.tar.gz: 39b333003a9c9d4fe5b211893fa3f286e41cd6c4
 SHA512:
-  metadata.gz: eebe0699ef7e179305c3af7b087e9b2d4f3e384b9d58789bb4547678ea9105d0eb6318a7c6473b520bb899b0c6f785c9315d01e9de976178722af5d12e2cce84
-  data.tar.gz: db62f865e8b7375e970d49ace20d983946d04aad8127c241e05b2ac87623c1b654d21de313086152fa38b9aae03140eb4c793f765b6ab4d865820303fe362b35
+  metadata.gz: 147fe43e230dc0eba2e3197622388ee79ca26e59aa111e4eabc484dd81eb66770e1d769434656e2bccef020dcddd7af3cf5487c05ead2e04b99fde37a86ffce8
+  data.tar.gz: 6bda16873866db44e312bfbafe985b1fac2b482fddb0b0c03a53838a7ba691e436601e9bb37d24aafcf1cb07dfdcce2d7fabd0464a321d9a50637f35afc3ae59

data/.gitignore

@@ -12,3 +12,4 @@ html
 pkg
 publish
 test/cache.tst
+tmp/

data/Contributing.rdoc

@@ -3,7 +3,7 @@
 I value any contribution to mime-types you can provide: a bug report, a feature
 request, or code contributions.
 
-As mime-types is a mature codebase, there are a few guidelines:
+There are a few guidelines for contributing to mime-types:
 
 * Code changes *will* *not* be accepted without tests. The test suite is
   written with {Minitest}[https://github.com/seattlerb/minitest].
@@ -18,9 +18,12 @@ As mime-types is a mature codebase, there are a few guidelines:
 
 === Adding or Modifying MIME Types
 
-The mime-types registry is loaded from a JSON file in +data+, but this file is
-not considered editable and cannot be compared in a pull request. New or
-modified MIME types should be edited in the appropriate YAML file under
+The mime-types registry is loaded from encoded files in +data+. These are not
+editable and cannot be compared meaningfully in a pull request; pull requests
+that include changes to these files will require amendment to revert these
+files.
+
+New or modified MIME types should be edited in the appropriate YAML file under
 +type-lists+. The format is as shown below for the +application/xml+ MIME type
 in +type-lists/application.yml+.
 
@@ -46,7 +49,20 @@ to verify that the JSON matches the YAML changes, which is why it is not
 necessary to convert for the pull request.
 
 If you are making a change for a private fork, use <tt>rake
-convert:yaml:json</tt> to convert the YAML to JSON.
+convert:yaml:json</tt> to convert the YAML to JSON, or <tt>rake
+convert:yaml:columnar</tt> to convert it to the new columnar format.
+
+==== Updating Types from the IANA or Apache Lists
+
+If you are maintaining a private fork and wish to update your copy of the MIME
+types registry used by this gem, you can do this with the rake tasks:
+
+    $ rake mime:iana
+    $ rake mime:apache
+
+Both of these require
+{Nokogiri}[http://www.nokogiri.org/tutorials/installing_nokogiri.html], which
+is not installed by default. Install it in the usual way for your Ruby.
 
 === Test Dependencies
 
@@ -77,12 +93,41 @@ can accomplish the same thing with:
 This task will install any missing dependencies, run the tests/specs, and
 generate the RDoc.
 
+You can run tests with code coverage analysis by running:
+
+    $ rake test:coverage
+
+=== Benchmarks
+
+mime-types offers several benchmark tasks to measure different measures of
+performance.
+
+There is a repeated load test, measuring how long it takes to start and load
+mime-types with its full registry. By default, it runs fifty loops and uses the
+built-in benchmark library.
+
+    $ rake benchmark:load
+
+There are two allocation tracing benchmarks (for normal and columnar loads).
+These can only be run on Ruby 2.1 or better and requires the
+{allocation_tracer}[https://github.com/ko1/allocation_tracer] gem (not
+installed by default).
+
+    $ rake benchmark:allocations
+    $ rake benchmark:allocations:columnar
+
+There are two loaded object count benchmarks (for normal and columnar loads).
+These use <tt>ObjectSpace.count_objects</tt>.
+
+    $ rake benchmark:objects
+    $ rake benchmark:objects:columnar
+
 === Workflow
 
 Here's the most direct way to get your work merged into the project:
 
 * Fork the project.
-* Clone down your fork (<tt>git clone git://github.com/<username>/mime-types.git</tt>).
+* Clone down your fork (<tt>git clone git://github.com/<username>/ruby-mime-types.git</tt>).
 * Create a topic branch to contain your change (<tt>git checkout -b my\_awesome\_feature</tt>).
 * Hack away, add tests. Not necessarily in that order.
 * Make sure everything still passes by running +rake+.
@@ -97,24 +142,31 @@ Here's the most direct way to get your work merged into the project:
 
 Thanks to everyone else who has contributed to mime-types:
 
+* Aaron Patterson
+* Aggelos Avgerinos
 * Andre Pankratz
-* Mauricio Linhares
-* Richard Hirner
-* Todd Carrico
+* Andy Brody
+* Arnaud Meuret
+* Brandon Galbraith
+* Chris Gat
+* David Genord
+* Eric Marden
 * Garret Alfert
+* Godfrey Chan
+* Greg Brockman
 * Hans de Graaff
 * Henrik Hodne
-* Martin d'Allens
-* Chris Gat
-* Greg Brockman
-* Łukasz Śliwa (for the friendly names)
+* Jeremy Evans
+* Juanito Fatas
+* Łukasz Śliwa
 * Keerthi Siva
-* Aaron Patterson
-* Godfrey Chan
-* Tibor Szolár
 * Ken Ip
-* David Genord
-* Juanito Fatas
-* Andy Brody
-* Aggelow Avgerinos
+* Martin d'Allens
+* Mauricio Linhares
+* nycvotes-dev
+* Postmodern
+* Richard Hirner
+* Richard Hurt
 * Richard Schneeman
+* Tibor Szolár
+* Todd Carrico

data/History-Types.rdoc

@@ -1,5 +1,16 @@
 = MIME Types Changes by Version
 
+== 2.6 / 2015-05-25
+
+* Steven Michael Thomas (@stevenmichaelthomas) added +woff2+ as an extension to
+  application/font-woff,
+  {#99}[https://github.com/mime-types/ruby-mime-types/pull/99].
+* Updated the IANA media registry entries as of release date:
+  * Updated metadata for application/jose, application/jose\+json,
+    application/jwk\+json, applicatoin/jwk-set\+json, application/jwt to
+    reflect the adoption of RFC7519.
+  * Added application/vnd.balsamiq.bmpr.
+
 == 2.5 / 2015-04-25
 
 * Updated the IANA media registry entries as of release date:

data/History.rdoc

@@ -1,7 +1,37 @@
+== 2.6 / 2015-05-25
+
+* New Feature:
+  * Columnar data storage for the MIME::Types registry, contributed by Jeremy
+    Evans (@jeremyevans). Reduces default memory use substantially (the mail
+    gem drops from 19 Mib to about 3 Mib). Resolves
+    {#96}[https://github.com/mime-types/ruby-mime-types/pull/96],
+    {#94}[https://github.com/mime-types/ruby-mime-types/issues/94],
+    {#83}[https://github.com/mime-types/ruby-mime-types/issues/83]. Partially
+    addresses {#64}[https://github.com/mime-types/ruby-mime-types/issues/64]
+    and {#62}[https://github.com/mime-types/ruby-mime-types/issues/62].
+* Development:
+  * Removed caching of deprecation messages in preparation for mime-types 3.0.
+    Now, deprecated methods will always warn their deprecation instead of only
+    warning once.
+  * Added a logger for deprecation messages.
+  * Renamed <tt>lib/mime.rb</tt> to <tt>lib/mime/deprecations.rb</tt> to not
+    conflict with the {mime}[https://rubygems.org/gems/mime] gem on behalf of
+    the maintainers of the {Praxis Framework}[http://praxis-framework.io/].
+    Provided by Josep M. Blanquer (@blanquer),
+    {#100}[https://github.com/mime-types/ruby-mime-types/pull/100].
+  * Added the columnar data conversion tool, also provided by Jeremy Evans.
+* Documentation:
+  * Improved documentation and ensured that all deprecated methods are marked
+    as such in the documentation.
+* Development:
+  * Added more Ruby variants to Travis CI.
+  * Silenced deprecation messages for internal tools. Noisy deprecations are
+    noisy, but that's the point.
+
 == 2.5 / 2015-04-25
 
 * Bugs:
-  * David Genord (@albus522) fixed a bug in loading MIME::types cache where a
+  * David Genord (@albus522) fixed a bug in loading MIME::Types cache where a
     container loaded from cache did not have the expected +default_proc+,
     {#86}[https://github.com/mime-types/ruby-mime-types/pull/86].
   * Richard Schneeman (@schneems) provided a patch that substantially reduces

data/Manifest.txt

@@ -10,18 +10,35 @@ Manifest.txt
 README.rdoc
 Rakefile
 data/mime-types.json
+data/mime.content_type.column
+data/mime.docs.column
+data/mime.encoding.column
+data/mime.friendly.column
+data/mime.obsolete.column
+data/mime.references.column
+data/mime.registered.column
+data/mime.signature.column
+data/mime.system.column
+data/mime.use_instead.column
+data/mime.xrefs.column
 docs/COPYING.txt
 docs/artistic.txt
 lib/mime-types.rb
-lib/mime.rb
 lib/mime/type.rb
+lib/mime/type/columnar.rb
 lib/mime/types.rb
 lib/mime/types/cache.rb
+lib/mime/types/columnar.rb
+lib/mime/types/deprecations.rb
 lib/mime/types/loader.rb
 lib/mime/types/loader_path.rb
+lib/mime/types/logger.rb
 support/apache_mime_types.rb
 support/benchmarks/load.rb
+support/benchmarks/load_allocations.rb
+support/benchmarks/object_counts.rb
 support/convert.rb
+support/convert/columnar.rb
 support/iana_registry.rb
 test/bad-fixtures/malformed
 test/fixture/json.json

data/README.rdoc

@@ -18,33 +18,43 @@ MIME content types are used in MIME-compliant communications, as in e-mail or
 HTTP traffic, to indicate the type of content which is transmitted. The
 mime-types library provides the ability for detailed information about MIME
 entities (provided as an enumerable collection of MIME::Type objects) to be
-determined and used programmatically. There are many types defined by RFCs and
-vendors, so the list is long but by definition incomplete; don't hesitate to
-add additional type definitions (see Contributing.rdoc). The primary sources
-for MIME type definitions found in mime-types is the
-{IANA Media Types registry}[https://www.iana.org/assignments/media-types/media-types.xhtml],
-RFCs, and W3C recommendations. It conforms to RFCs 2045 and 2231.
-
-This is release 2.5 with a couple of bug fixes, updating to the latest IANA
-type registry, and adding a user-contributed type. mime-types 2.x supports Ruby
-1.9.2 or later.
+determined and used. There are many types defined by RFCs and vendors, so the
+list is long but by definition incomplete; don't hesitate to add additional
+type definitions. MIME type definitions found in mime-types are from RFCs, W3C
+recommendations, the {IANA Media Types
+registry}[https://www.iana.org/assignments/media-types/media-types.xhtml], and
+user contributions. It conforms to RFCs 2045 and 2231.
+
+This is release 2.6 with two new experimental features. The first new feature
+is a new default registry storage format that greatly reduces the initial
+memory use of the mime-types library. This feature is enabled by requiring
++mime/types/columnar+ instead of +mime/types+ with a small performance cost and
+no change in *total* memory use if certain methods are called (see {Columnar
+Store}[#columnar-store] for more details). The second new feature is a logger
+interface that conforms to the expectations of an ActiveSupport::Logger so that
+warnings can be written to an application's log rather than the default
+location for +warn+. This interface may be used for other logging purposes in
+the future.
+
+mime-types 2.6 is the last planned version of mime-types 2.x, so deprecation
+warnings are no longer cached but provided every time the method is called.
+mime-types 2.6 supports Ruby 1.9.2 or later.
 
 === mime-types 1.x End of Life
 
 mime-types 2.0 was released in late 2013, and as of early 2015 there have been
 no reported security issues for mime-types 1.x. With the release of mime-types
-2.5, I setting the formal End of Life for mime-types 1.x for 2015-10-27 (the
-second anniversary of the release of mime-types 2.0). After this date,
-absolutely no pull requests for mime-types 1.x will be accepted.
+2.5, I set the formal End of Life for mime-types 1.x for 2015-10-27 (the second
+anniversary of the release of mime-types 2.0). After this date, absolutely no
+pull requests for mime-types 1.x will be accepted.
 
 === mime-types Future
 
-Even though there are a number of issues open, it is clear to me that there are
-some fundamental changes that need to happen to both the data representation
-and the API provided by mime-types. This cannot happen under the current
-release, so all new development is focussing on an upcoming 3.0 release. The
-target for the release is on or before the beginning of RubyConf 2015
-(2015-11-15).
+There are a number of issues open that make clear to me that there are some
+fundamental changes that need to happen to both the data representation and the
+API provided by mime-types. This cannot happen under the current release, so
+all new development is focussing on an upcoming 3.0 release. The target for the
+release is on or before the beginning of RubyConf 2015 (2015-11-15).
 
 When 3.0 is released, mime-types 2.x will receive regular updates of the IANA
 registry for two years following the release. It will also receive security
@@ -52,9 +62,15 @@ updates, if needed, for the same period. There will be no further feature
 development on mime-types 2.x following the 3.0 release.
 
 Coincident with the 3.0 release, I will release mime-types 2.99.0 that no
-longer imports the data to fields that have been deprecated. If they work
-because they derive data from that which is imported, they will continue to
-work. The quarterly updates will be against 2.99.x.
+longer imports the data to fields that have been deprecated, or exports it if
+it is present. If they work because they derive data from the data that is
+still present, the will continue to work. The quarterly updates will be against
+2.99.x.
+
+If the possible loss of this deprecated data matters, be sure to set your
+dependency appropriately:
+
+   gem 'mime-types', '~> 2.6, < 2.99'
 
 == Synopsis
 
@@ -62,44 +78,109 @@ MIME types are used in MIME entities, as in email or HTTP traffic. It is useful
 at times to have information available about MIME types (or, inversely, about
 files). A MIME::Type stores the known information about one MIME type.
 
-    require 'mime/types'
-
-    plaintext = MIME::Types['text/plain'] # => [ text/plain ]
-    text = plaintext.first
-    puts text.media_type            # => 'text'
-    puts text.sub_type              # => 'plain'
-
-    puts text.extensions.join(" ")  # => 'txt asc c cc h hh cpp hpp dat hlp'
-    puts text.preferred_extension   # => 'txt'
-    puts text.friendly              # => 'Text Document'
-    puts text.i18n_key              # => 'text.plain'
-
-    puts text.encoding              # => quoted-printable
-    puts text.binary?               # => false
-    puts text.ascii?                # => true
-    puts text.obsolete?             # => false
-    puts text.registered?           # => true
-    puts text == 'text/plain'       # => true
-    puts 'text/plain' == text       # => true
-    puts MIME::Type.simplified('x-appl/x-zip')
-                                    # => 'appl/zip'
-
-    puts MIME::Types.any? { |type|
-      type.content_type == 'text/plain'
-    }                               # => true
-    puts MIME::Types.all?(&:registered?)
-                                    # => false
-
-    # Various string representations of MIME types
-
-    qcelp = MIME::Types['audio/QCELP'].first # => audio/QCELP
-    puts qcelp.content_type         # => 'audio/QCELP'
-    puts qcelp.simplified           # => 'audio/qcelp'
-
-    xwingz = MIME::Types['application/x-Wingz'].first # => application/x-Wingz
-    puts xwingz.content_type        # => 'application/x-Wingz'
-    puts xwingz.simplified          # => 'application/wingz'
-
+   require 'mime/types'
+
+   plaintext = MIME::Types['text/plain'] # => [ text/plain ]
+   text = plaintext.first
+   puts text.media_type            # => 'text'
+   puts text.sub_type              # => 'plain'
+
+   puts text.extensions.join(' ')  # => 'txt asc c cc h hh cpp hpp dat hlp'
+   puts text.preferred_extension   # => 'txt'
+   puts text.friendly              # => 'Text Document'
+   puts text.i18n_key              # => 'text.plain'
+
+   puts text.encoding              # => quoted-printable
+   puts text.default_encoding      # => quoted-printable
+   puts text.binary?               # => false
+   puts text.ascii?                # => true
+   puts text.obsolete?             # => false
+   puts text.registered?           # => true
+   puts text.complete?             # => true
+
+   puts text                       # => 'text/plain'
+
+   puts text == 'text/plain'       # => true
+   puts 'text/plain' == text       # => true
+   puts text == 'text/x-plain'     # => false
+   puts 'text/x-plain' == text     # => false
+
+   puts MIME::Type.simplified('x-appl/x-zip') # => 'appl/zip'
+   puts MIME::Type.i18n_key('x-appl/x-zip') # => 'appl.zip'
+
+   puts text.like?('text/x-plain') # => true
+   puts text.like?(MIME::Type.new('x-text/x-plain')) # => true
+
+   puts text.xrefs.inspect # => { "rfc" => [ "rfc2046", "rfc3676", "rfc5147" ] }
+   puts text.urls # => [ "http://www.iana.org/go/rfc2046",
+                  #      "http://www.iana.org/go/rfc3676",
+                  #      "http://www.iana.org/go/rfc5147" ]
+
+   xtext = MIME::Type.new('x-text/x-plain')
+   puts xtext.media_type # => 'text'
+   puts xtext.raw_media_type # => 'x-text'
+   puts xtext.sub_type # => 'plain'
+   puts xtext.raw_sub_type # => 'x-plain'
+   puts xtext.complete? # => false
+
+   puts MIME::Types.any? { |type| type.content_type == 'text/plain' } # => true
+   puts MIME::Types.all?(&:registered?) # => false
+
+   # Various string representations of MIME types
+   qcelp = MIME::Types['audio/QCELP'].first # => audio/QCELP
+   puts qcelp.content_type         # => 'audio/QCELP'
+   puts qcelp.simplified           # => 'audio/qcelp'
+
+   xwingz = MIME::Types['application/x-Wingz'].first # => application/x-Wingz
+   puts xwingz.content_type        # => 'application/x-Wingz'
+   puts xwingz.simplified          # => 'application/wingz'
+
+=== Columnar Store
+
+mime-types 2.6 has an experimental columnar storage format that reduces the
+default memory footprint. It does this by selectively loading data. When a
+registry is first loaded from a columnar store, only the canonical MIME type
+and registered extensions will be loaded and the MIME type will be connected to
+its registry. When extended data is required (including #registered, #obsolete,
+#use_instead), that data is loaded from its own column file for all types in
+the registry. This load is done with a Mutex to ensure that the types are
+updated safely in a multithreaded environment.
+
+Columnar storage is slated to become the default storage format for mime-types
+3.0, but until that is released, the default is still to use the JSON storage
+format. As such, columnar storage can only currently be loaded at an
+application level with the following specification in the application Gemfile:
+
+   gem 'mime-types', require: 'mime/types/columnar'
+
+Projects that do not use Bundler, and libraries that wish to suggest this
+behaviour to applications are encouraged to require this directly, but only if
+you specify a dependency on mime-types 2.6.
+
+   require 'mime/types/columnar'
+
+Although this require will not be necessary after mime-types 3, it will work
+through at least {version
+4}[https://github.com/mime-types/ruby-mime-types/pull/96#issuecomment-100725400]
+and possibly beyond.
+
+Note that the new Columnar class (MIME::Type::Columnar) and module
+(MIME::Types::Columnar) are considered private variant implementations of
+MIME::Type and MIME::Types and the specific implementation should not be relied
+upon by consumers of the mime-types library. Instead, depend on the public
+implementations only.
+
+=== Cached Storage
+
+Since version 2.0, mime-types has supported a cache of MIME types based on
+<tt>Marshal.dump</tt>. The cache is invalidated for each released version of
+mime-types so that version 2.5 is not reused for version 2.6. If the
+environment variable +RUBY_MIME_TYPES_CACHE+ is set to a cache file, mime-types
+will attempt to load the MIME type registry from the cache file. If it cannot,
+it will load the types normally and then saves the registry to the cache file.
+
+The current mime-types cache is not compatible with the columnar storage
+format. This will be resolved for mime-types 3.
 
 == mime-types Modified Semantic Versioning
 
@@ -129,7 +210,7 @@ In practical terms, there should be a MINOR release roughly monthly to track
 updated or changed MIME types from the official IANA registry. This does not
 indicate when new API features have been added, but all minor versions of
 mime-types 2.x will be backwards compatible; the interfaces marked deprecated
-will not be removed until at least mime-types 3.x or possibly later.
+will be removed in mime-types 3.x.
 
 :include: Contributing.rdoc