hane 1.1.1 → 1.2.0

package/vendor/zlib/doc/txtvsbin.txt

@@ -1,107 +1,107 @@
-A Fast Method for Identifying Plain Text Files
-==============================================
-
-
-Introduction
-------------
-
-Given a file coming from an unknown source, it is sometimes desirable
-to find out whether the format of that file is plain text.  Although
-this may appear like a simple task, a fully accurate detection of the
-file type requires heavy-duty semantic analysis on the file contents.
-It is, however, possible to obtain satisfactory results by employing
-various heuristics.
-
-Previous versions of PKZip and other zip-compatible compression tools
-were using a crude detection scheme: if more than 80% (4/5) of the bytes
-found in a certain buffer are within the range [7..127], the file is
-labeled as plain text, otherwise it is labeled as binary.  A prominent
-limitation of this scheme is the restriction to Latin-based alphabets.
-Other alphabets, like Greek, Cyrillic or Asian, make extensive use of
-the bytes within the range [128..255], and texts using these alphabets
-are most often misidentified by this scheme; in other words, the rate
-of false negatives is sometimes too high, which means that the recall
-is low.  Another weakness of this scheme is a reduced precision, due to
-the false positives that may occur when binary files containing large
-amounts of textual characters are misidentified as plain text.
-
-In this article we propose a new, simple detection scheme that features
-a much increased precision and a near-100% recall.  This scheme is
-designed to work on ASCII, Unicode and other ASCII-derived alphabets,
-and it handles single-byte encodings (ISO-8859, MacRoman, KOI8, etc.)
-and variable-sized encodings (ISO-2022, UTF-8, etc.).  Wider encodings
-(UCS-2/UTF-16 and UCS-4/UTF-32) are not handled, however.
-
-
-The Algorithm
--------------
-
-The algorithm works by dividing the set of bytecodes [0..255] into three
-categories:
-- The allow list of textual bytecodes:
-  9 (TAB), 10 (LF), 13 (CR), 32 (SPACE) to 255.
-- The gray list of tolerated bytecodes:
-  7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB), 27 (ESC).
-- The block list of undesired, non-textual bytecodes:
-  0 (NUL) to 6, 14 to 31.
-
-If a file contains at least one byte that belongs to the allow list and
-no byte that belongs to the block list, then the file is categorized as
-plain text; otherwise, it is categorized as binary.  (The boundary case,
-when the file is empty, automatically falls into the latter category.)
-
-
-Rationale
----------
-
-The idea behind this algorithm relies on two observations.
-
-The first observation is that, although the full range of 7-bit codes
-[0..127] is properly specified by the ASCII standard, most control
-characters in the range [0..31] are not used in practice.  The only
-widely-used, almost universally-portable control codes are 9 (TAB),
-10 (LF) and 13 (CR).  There are a few more control codes that are
-recognized on a reduced range of platforms and text viewers/editors:
-7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB) and 27 (ESC); but these
-codes are rarely (if ever) used alone, without being accompanied by
-some printable text.  Even the newer, portable text formats such as
-XML avoid using control characters outside the list mentioned here.
-
-The second observation is that most of the binary files tend to contain
-control characters, especially 0 (NUL).  Even though the older text
-detection schemes observe the presence of non-ASCII codes from the range
-[128..255], the precision rarely has to suffer if this upper range is
-labeled as textual, because the files that are genuinely binary tend to
-contain both control characters and codes from the upper range.  On the
-other hand, the upper range needs to be labeled as textual, because it
-is used by virtually all ASCII extensions.  In particular, this range is
-used for encoding non-Latin scripts.
-
-Since there is no counting involved, other than simply observing the
-presence or the absence of some byte values, the algorithm produces
-consistent results, regardless what alphabet encoding is being used.
-(If counting were involved, it could be possible to obtain different
-results on a text encoded, say, using ISO-8859-16 versus UTF-8.)
-
-There is an extra category of plain text files that are "polluted" with
-one or more block-listed codes, either by mistake or by peculiar design
-considerations.  In such cases, a scheme that tolerates a small fraction
-of block-listed codes would provide an increased recall (i.e. more true
-positives).  This, however, incurs a reduced precision overall, since
-false positives are more likely to appear in binary files that contain
-large chunks of textual data.  Furthermore, "polluted" plain text should
-be regarded as binary by general-purpose text detection schemes, because
-general-purpose text processing algorithms might not be applicable.
-Under this premise, it is safe to say that our detection method provides
-a near-100% recall.
-
-Experiments have been run on many files coming from various platforms
-and applications.  We tried plain text files, system logs, source code,
-formatted office documents, compiled object code, etc.  The results
-confirm the optimistic assumptions about the capabilities of this
-algorithm.
-
-
---
-Cosmin Truta
-Last updated: 2006-May-28
+A Fast Method for Identifying Plain Text Files
+==============================================
+
+
+Introduction
+------------
+
+Given a file coming from an unknown source, it is sometimes desirable
+to find out whether the format of that file is plain text.  Although
+this may appear like a simple task, a fully accurate detection of the
+file type requires heavy-duty semantic analysis on the file contents.
+It is, however, possible to obtain satisfactory results by employing
+various heuristics.
+
+Previous versions of PKZip and other zip-compatible compression tools
+were using a crude detection scheme: if more than 80% (4/5) of the bytes
+found in a certain buffer are within the range [7..127], the file is
+labeled as plain text, otherwise it is labeled as binary.  A prominent
+limitation of this scheme is the restriction to Latin-based alphabets.
+Other alphabets, like Greek, Cyrillic or Asian, make extensive use of
+the bytes within the range [128..255], and texts using these alphabets
+are most often misidentified by this scheme; in other words, the rate
+of false negatives is sometimes too high, which means that the recall
+is low.  Another weakness of this scheme is a reduced precision, due to
+the false positives that may occur when binary files containing large
+amounts of textual characters are misidentified as plain text.
+
+In this article we propose a new, simple detection scheme that features
+a much increased precision and a near-100% recall.  This scheme is
+designed to work on ASCII, Unicode and other ASCII-derived alphabets,
+and it handles single-byte encodings (ISO-8859, MacRoman, KOI8, etc.)
+and variable-sized encodings (ISO-2022, UTF-8, etc.).  Wider encodings
+(UCS-2/UTF-16 and UCS-4/UTF-32) are not handled, however.
+
+
+The Algorithm
+-------------
+
+The algorithm works by dividing the set of bytecodes [0..255] into three
+categories:
+- The allow list of textual bytecodes:
+  9 (TAB), 10 (LF), 13 (CR), 32 (SPACE) to 255.
+- The gray list of tolerated bytecodes:
+  7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB), 27 (ESC).
+- The block list of undesired, non-textual bytecodes:
+  0 (NUL) to 6, 14 to 31.
+
+If a file contains at least one byte that belongs to the allow list and
+no byte that belongs to the block list, then the file is categorized as
+plain text; otherwise, it is categorized as binary.  (The boundary case,
+when the file is empty, automatically falls into the latter category.)
+
+
+Rationale
+---------
+
+The idea behind this algorithm relies on two observations.
+
+The first observation is that, although the full range of 7-bit codes
+[0..127] is properly specified by the ASCII standard, most control
+characters in the range [0..31] are not used in practice.  The only
+widely-used, almost universally-portable control codes are 9 (TAB),
+10 (LF) and 13 (CR).  There are a few more control codes that are
+recognized on a reduced range of platforms and text viewers/editors:
+7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB) and 27 (ESC); but these
+codes are rarely (if ever) used alone, without being accompanied by
+some printable text.  Even the newer, portable text formats such as
+XML avoid using control characters outside the list mentioned here.
+
+The second observation is that most of the binary files tend to contain
+control characters, especially 0 (NUL).  Even though the older text
+detection schemes observe the presence of non-ASCII codes from the range
+[128..255], the precision rarely has to suffer if this upper range is
+labeled as textual, because the files that are genuinely binary tend to
+contain both control characters and codes from the upper range.  On the
+other hand, the upper range needs to be labeled as textual, because it
+is used by virtually all ASCII extensions.  In particular, this range is
+used for encoding non-Latin scripts.
+
+Since there is no counting involved, other than simply observing the
+presence or the absence of some byte values, the algorithm produces
+consistent results, regardless what alphabet encoding is being used.
+(If counting were involved, it could be possible to obtain different
+results on a text encoded, say, using ISO-8859-16 versus UTF-8.)
+
+There is an extra category of plain text files that are "polluted" with
+one or more block-listed codes, either by mistake or by peculiar design
+considerations.  In such cases, a scheme that tolerates a small fraction
+of block-listed codes would provide an increased recall (i.e. more true
+positives).  This, however, incurs a reduced precision overall, since
+false positives are more likely to appear in binary files that contain
+large chunks of textual data.  Furthermore, "polluted" plain text should
+be regarded as binary by general-purpose text detection schemes, because
+general-purpose text processing algorithms might not be applicable.
+Under this premise, it is safe to say that our detection method provides
+a near-100% recall.
+
+Experiments have been run on many files coming from various platforms
+and applications.  We tried plain text files, system logs, source code,
+formatted office documents, compiled object code, etc.  The results
+confirm the optimistic assumptions about the capabilities of this
+algorithm.
+
+
+--
+Cosmin Truta
+Last updated: 2006-May-28

package/vendor/zlib/examples/README.examples

@@ -1,54 +1,54 @@
-This directory contains examples of the use of zlib and other relevant
-programs and documentation.
-
-enough.c
-    calculation and justification of ENOUGH parameter in inftrees.h
-    - calculates the maximum table space used in inflate tree
-      construction over all possible Huffman codes
-
-fitblk.c
-    compress just enough input to nearly fill a requested output size
-    - zlib isn't designed to do this, but fitblk does it anyway
-
-gun.c
-    uncompress a gzip file
-    - illustrates the use of inflateBack() for high speed file-to-file
-      decompression using call-back functions
-    - is approximately twice as fast as gzip -d
-    - also provides Unix uncompress functionality, again twice as fast
-
-gzappend.c
-    append to a gzip file
-    - illustrates the use of the Z_BLOCK flush parameter for inflate()
-    - illustrates the use of deflatePrime() to start at any bit
-
-gzjoin.c
-    join gzip files without recalculating the crc or recompressing
-    - illustrates the use of the Z_BLOCK flush parameter for inflate()
-    - illustrates the use of crc32_combine()
-
-gzlog.c
-gzlog.h
-    efficiently and robustly maintain a message log file in gzip format
-    - illustrates use of raw deflate, Z_PARTIAL_FLUSH, deflatePrime(),
-      and deflateSetDictionary()
-    - illustrates use of a gzip header extra field
-
-gznorm.c
-    normalize a gzip file by combining members into a single member
-    - demonstrates how to concatenate deflate streams using Z_BLOCK
-
-zlib_how.html
-    painfully comprehensive description of zpipe.c (see below)
-    - describes in excruciating detail the use of deflate() and inflate()
-
-zpipe.c
-    reads and writes zlib streams from stdin to stdout
-    - illustrates the proper use of deflate() and inflate()
-    - deeply commented in zlib_how.html (see above)
-
-zran.c
-zran.h
-    index a zlib or gzip stream and randomly access it
-    - illustrates the use of Z_BLOCK, inflatePrime(), and
-      inflateSetDictionary() to provide random access
+This directory contains examples of the use of zlib and other relevant
+programs and documentation.
+
+enough.c
+    calculation and justification of ENOUGH parameter in inftrees.h
+    - calculates the maximum table space used in inflate tree
+      construction over all possible Huffman codes
+
+fitblk.c
+    compress just enough input to nearly fill a requested output size
+    - zlib isn't designed to do this, but fitblk does it anyway
+
+gun.c
+    uncompress a gzip file
+    - illustrates the use of inflateBack() for high speed file-to-file
+      decompression using call-back functions
+    - is approximately twice as fast as gzip -d
+    - also provides Unix uncompress functionality, again twice as fast
+
+gzappend.c
+    append to a gzip file
+    - illustrates the use of the Z_BLOCK flush parameter for inflate()
+    - illustrates the use of deflatePrime() to start at any bit
+
+gzjoin.c
+    join gzip files without recalculating the crc or recompressing
+    - illustrates the use of the Z_BLOCK flush parameter for inflate()
+    - illustrates the use of crc32_combine()
+
+gzlog.c
+gzlog.h
+    efficiently and robustly maintain a message log file in gzip format
+    - illustrates use of raw deflate, Z_PARTIAL_FLUSH, deflatePrime(),
+      and deflateSetDictionary()
+    - illustrates use of a gzip header extra field
+
+gznorm.c
+    normalize a gzip file by combining members into a single member
+    - demonstrates how to concatenate deflate streams using Z_BLOCK
+
+zlib_how.html
+    painfully comprehensive description of zpipe.c (see below)
+    - describes in excruciating detail the use of deflate() and inflate()
+
+zpipe.c
+    reads and writes zlib streams from stdin to stdout
+    - illustrates the proper use of deflate() and inflate()
+    - deeply commented in zlib_how.html (see above)
+
+zran.c
+zran.h
+    index a zlib or gzip stream and randomly access it
+    - illustrates the use of Z_BLOCK, inflatePrime(), and
+      inflateSetDictionary() to provide random access