ruby-xz 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/AUTHORS +8 -0
- data/COPYING +1 -1
- data/HISTORY.rdoc +24 -4
- data/{README.rdoc → README.md} +61 -33
- data/lib/xz.rb +290 -184
- data/lib/xz/lib_lzma.rb +28 -18
- data/lib/xz/stream.rb +24 -22
- data/lib/xz/stream_reader.rb +328 -122
- data/lib/xz/stream_writer.rb +288 -97
- data/lib/xz/version.rb +33 -0
- metadata +36 -46
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 28c451853871a094353485f28db08d64c5a5effa
|
4
|
+
data.tar.gz: 93ef5cba40e6a6330364206caf86dbcca6f95a7e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 75552a3ee976a8441ff397953bd608a8e9e6e5508b9c7cae73eb05278f509475c00fcea7ade557f758520c5b06b42711d96c162e9e5f2f5cc7ebed05e810a99f
|
7
|
+
data.tar.gz: 11b58af7569d56fe0c7486945a9e5ef87bfb0bf9579d29a82c3f7f21f8aa9ae2d03e118927bdc295585e857b82d5d536ddb9012393646591dc4a58585462587e
|
data/AUTHORS
ADDED
data/COPYING
CHANGED
data/HISTORY.rdoc
CHANGED
@@ -1,16 +1,36 @@
|
|
1
|
-
|
1
|
+
= Version history
|
2
|
+
|
3
|
+
== 0.2.2 (unreleased)
|
4
|
+
|
5
|
+
* *Add* XZ.disable_deprecation_notes
|
6
|
+
* *Deprecate* use of XZ::StreamReader.open with an IO argument
|
7
|
+
* *Deprecate* use of XZ::StreamReader.new with a filename argument
|
8
|
+
* *Deprecate* use of XZ::StreamWriter.open with an IO argument
|
9
|
+
* *Deprecate* use of XZ::StreamWriter.new with a filename argument
|
10
|
+
* *Deprecate* nonautomatic IO close in XZ::StreamReader#close
|
11
|
+
* *Deprecate* nonautomatic IO close in XZ::StreamWriter#close
|
12
|
+
* *Fix* incompatibility with Resolv.getaddress() in Ruby 2.2 (Ticket #13
|
13
|
+
by Ken Simon)
|
14
|
+
* Goal of these deprecations is to sync the API with Ruby’s own
|
15
|
+
Zlib::GzipWriter and Zlib::GzipReader mostly.
|
16
|
+
* Add required versions to gemspec.
|
17
|
+
* Comment format cleanup, results in better docs.
|
18
|
+
* Internal code cleanup
|
19
|
+
* Add more tests.
|
20
|
+
|
21
|
+
== 0.2.1 (2014-02-08)
|
2
22
|
|
3
23
|
* Build the gem properly on Ruby 2.0+ (PR #8 by Nana Sakisaka (saki7))
|
4
24
|
* Release the GIL when interfacing with liblzma (PR #7 by Lars Christensen (larsch))
|
5
25
|
|
6
|
-
== 0.2.0
|
26
|
+
== 0.2.0 (2013-06-23)
|
7
27
|
|
8
28
|
* Fix #6 (errors on JRuby) by Ben Nagy
|
9
29
|
* <b>Remove 1.8 compatibility</b>
|
10
30
|
|
11
|
-
== 0.1.0
|
31
|
+
== 0.1.0 (2013-02-17)
|
12
32
|
|
13
33
|
* <b>Add XZ::StreamReader and XZ::StreamWriter for io-like behaviour.</b>
|
14
34
|
* New dependency on the +io-like+ gem.
|
15
35
|
* <b>Add Ruby 1.8 compatibility.</b> Thanks to Christoph Plank.
|
16
|
-
* We now have proper unit tests.
|
36
|
+
* We now have proper unit tests.
|
data/{README.rdoc → README.md}
RENAMED
@@ -1,11 +1,12 @@
|
|
1
|
-
|
1
|
+
ruby-xz
|
2
|
+
=======
|
2
3
|
|
3
|
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
4
|
+
**ruby-xz** is a basic binding to the famous [liblzma library][1],
|
5
|
+
best known for the extreme compression-ratio it's native *XZ* format
|
6
|
+
achieves. ruby-xz gives you the possibility of creating and extracting
|
7
|
+
XZ archives on any platform where liblzma is installed. No compilation
|
8
|
+
is needed, because ruby-xz is written ontop of
|
9
|
+
[ffi][2].
|
9
10
|
|
10
11
|
ruby-xz supports both "intuitive" (de)compression by providing methods to
|
11
12
|
directly operate on strings and files, but also allows you to operate
|
@@ -14,59 +15,82 @@ of that, ruby-xz offers an advanced interface that allows you to treat
|
|
14
15
|
XZ-compressed data as IO streams, both for reading and for writing. See the
|
15
16
|
XZ::StreamReader and XZ::StreamWriter classes for more information on this.
|
16
17
|
|
17
|
-
|
18
|
+
Installation
|
19
|
+
------------
|
18
20
|
|
19
21
|
Install it the way you install all your gems.
|
20
22
|
|
21
|
-
|
23
|
+
```
|
24
|
+
$ gem install ruby-xz
|
25
|
+
```
|
22
26
|
|
23
|
-
|
27
|
+
Alternatively, you can clone the repository and build the most recent
|
28
|
+
code yourself:
|
24
29
|
|
25
|
-
|
30
|
+
```
|
31
|
+
$ git clone git://github.com/Quintus/ruby-xz.git
|
32
|
+
$ cd ruby-xz
|
33
|
+
$ rake gem
|
34
|
+
$ gem install pkg/ruby-xz-*.gem
|
35
|
+
```
|
36
|
+
|
37
|
+
Usage
|
38
|
+
-----
|
26
39
|
|
27
40
|
The documentation of the XZ module is well and you should be able to find
|
28
41
|
everything you need to use ruby-xz. As said, it's not big, but powerful:
|
29
42
|
You can create and extract whole archive files, compress or decompress
|
30
43
|
streams of data or just plain strings.
|
31
44
|
|
32
|
-
You can read the documentation on your local gemserver, or online
|
33
|
-
http://quintus.github.io/ruby-xz/.
|
45
|
+
You can read the documentation on your local gemserver, or browse it [online][3].
|
34
46
|
|
35
|
-
|
47
|
+
### First step ###
|
36
48
|
|
37
49
|
You have to require ruby-xz. Note the file you have to require is named
|
38
50
|
"xz.rb", so do
|
39
51
|
|
40
|
-
|
41
|
-
|
52
|
+
``` ruby
|
53
|
+
require "xz"
|
54
|
+
```
|
55
|
+
|
42
56
|
to get it.
|
43
57
|
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
58
|
+
### Examples ###
|
59
|
+
|
60
|
+
``` ruby
|
61
|
+
# Compress a TAR archive
|
62
|
+
XZ.compress_file("myfile.tar", "myfile.tar.xz")
|
63
|
+
# Decompress it
|
64
|
+
XZ.decompress_file("myfile.tar.xz", "myfile.tar")
|
65
|
+
|
66
|
+
# Compress everything you get from a socket (note that there HAS to be a EOF
|
67
|
+
# sometime, otherwise this will run infinitely)
|
68
|
+
XZ.compress_stream(socket){|chunk| opened_file.write(chunk)}
|
69
|
+
|
70
|
+
# Compress a string
|
71
|
+
comp = XZ.compress("Mydata")
|
72
|
+
# Decompress it
|
73
|
+
data = XZ.decompress(comp)
|
74
|
+
```
|
59
75
|
|
60
76
|
Have a look at the XZ module's documentation for an in-depth description of
|
61
77
|
what is possible.
|
62
78
|
|
63
|
-
|
79
|
+
Links
|
80
|
+
-----
|
81
|
+
|
82
|
+
* Code repository: https://github.com/Quintus/ruby-xz
|
83
|
+
* Issue tracker: https://github.com/Quintus/ruby-xz/issues
|
84
|
+
* Online documentation: http://quintus.github.io/ruby-xz
|
85
|
+
|
86
|
+
License
|
87
|
+
-------
|
64
88
|
|
65
89
|
(The MIT License)
|
66
90
|
|
67
91
|
Basic liblzma-bindings for Ruby.
|
68
92
|
|
69
|
-
Copyright © 2011-
|
93
|
+
Copyright © 2011-2015 Marvin Gülker et al.
|
70
94
|
|
71
95
|
See AUTHORS for the full list of contributors.
|
72
96
|
|
@@ -87,3 +111,7 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
87
111
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
88
112
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
89
113
|
THE SOFTWARE.
|
114
|
+
|
115
|
+
[1]: http://tukaani.org/xz/
|
116
|
+
[2]: https://github.com/ffi/ffi
|
117
|
+
[3]: http://quintus.github.io/ruby-xz
|
data/lib/xz.rb
CHANGED
@@ -1,9 +1,10 @@
|
|
1
1
|
# -*- coding: utf-8 -*-
|
2
|
+
#--
|
2
3
|
# (The MIT License)
|
3
4
|
#
|
4
5
|
# Basic liblzma-bindings for Ruby.
|
5
6
|
#
|
6
|
-
# Copyright © 2011,2012 Marvin Gülker
|
7
|
+
# Copyright © 2011,2012,2015 Marvin Gülker
|
7
8
|
# Copyright © 2011 Christoph Plank
|
8
9
|
#
|
9
10
|
# Permission is hereby granted, free of charge, to any person obtaining a
|
@@ -23,79 +24,122 @@
|
|
23
24
|
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
24
25
|
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
25
26
|
# THE SOFTWARE.
|
27
|
+
#++
|
26
28
|
|
27
29
|
require "pathname"
|
28
30
|
require "ffi"
|
29
31
|
require 'stringio'
|
30
32
|
require "io/like"
|
31
33
|
|
32
|
-
#The namespace and main module of this library. Each method of this
|
33
|
-
#may raise exceptions of class XZ::LZMAError, which is not
|
34
|
-
#methods' documentations anymore.
|
34
|
+
# The namespace and main module of this library. Each method of this
|
35
|
+
# module may raise exceptions of class XZ::LZMAError, which is not
|
36
|
+
# named in the methods' documentations anymore.
|
35
37
|
#
|
36
|
-
#All strings you receive from any method defined in this module
|
37
|
-
#
|
38
|
-
#
|
39
|
-
#
|
40
|
-
#ruby-xz can’t handle this as compiled strings don’t come with
|
41
|
-
#information.
|
38
|
+
# All strings you receive from any method defined in this module and
|
39
|
+
# the classes defined in it are encoded in BINARY, so you may have to
|
40
|
+
# call #force_encoding on them to tag them with the correct encoding
|
41
|
+
# (assuming you _know_ what their correct encoding should be).
|
42
|
+
# ruby-xz can’t handle this as compiled strings don’t come with
|
43
|
+
# encoding information.
|
42
44
|
module XZ
|
43
|
-
#The version of this library.
|
44
|
-
VERSION = "0.2.1"
|
45
45
|
|
46
|
-
#Number of bytes read in one chunk.
|
46
|
+
# Number of bytes read in one chunk.
|
47
47
|
CHUNK_SIZE = 4096
|
48
48
|
|
49
49
|
class << self
|
50
50
|
|
51
|
-
#
|
52
|
-
#
|
53
|
-
#
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
#
|
63
|
-
#
|
64
|
-
#
|
65
|
-
#
|
66
|
-
#
|
67
|
-
#
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
#
|
73
|
-
|
74
|
-
#
|
75
|
-
#
|
76
|
-
#
|
77
|
-
#
|
78
|
-
#
|
79
|
-
#
|
80
|
-
#
|
81
|
-
|
82
|
-
#
|
83
|
-
#
|
84
|
-
#
|
85
|
-
#
|
86
|
-
#
|
51
|
+
# :nodoc:
|
52
|
+
#
|
53
|
+
# Output a deprecation notice.
|
54
|
+
def deprecate(msg)
|
55
|
+
@disable_deprecation_notices ||= false
|
56
|
+
|
57
|
+
unless @disable_deprecation_notices
|
58
|
+
$stderr.puts("DEPRECATION NOTICE: #{msg}\n#{caller.drop(1).join("\n\t")}")
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
# Force ruby-xz to be silent about deprecations. Using this is
|
63
|
+
# discouraged so that you are aware of upcoming changes to the
|
64
|
+
# API. However, if your standard error stream is closed,
|
65
|
+
# outputting the deprecation notices might result in an exception,
|
66
|
+
# so this method allows you to surpress these notices. Ensure you
|
67
|
+
# read the HISTORY.rdoc file carefully instead.
|
68
|
+
def disable_deprecation_notices=(bool)
|
69
|
+
@disable_deprecation_notices = bool
|
70
|
+
end
|
71
|
+
|
72
|
+
# call-seq:
|
73
|
+
# decompress_stream(io [, memory_limit [, flags ] ] ) → a_string
|
74
|
+
# decompress_stream(io [, memory_limit [, flags ] ] ){|chunk| ... } → an_integer
|
75
|
+
# decode_stream(io [, memory_limit [, flags ] ] ) → a_string
|
76
|
+
# decode_stream(io [, memory_limit [, flags ] ] ){|chunk| ... } → an_integer
|
77
|
+
#
|
78
|
+
# Decompresses a stream containing XZ-compressed data.
|
79
|
+
#
|
80
|
+
# === Parameters
|
81
|
+
#
|
82
|
+
# [io]
|
83
|
+
# The IO to read from. It must be opened for reading.
|
84
|
+
#
|
85
|
+
# [memory_limit (+UINT64_MAX+)]
|
86
|
+
# If not XZ::LibLZMA::UINT64_MAX, makes liblzma
|
87
|
+
# use no more memory than +memory_limit+ bytes.
|
88
|
+
#
|
89
|
+
# [flags (<tt>[:tell_unsupported_check]</tt>)]
|
90
|
+
# Additional flags
|
91
|
+
# passed to liblzma (an array). Possible flags are:
|
92
|
+
#
|
93
|
+
# [:tell_no_check]
|
94
|
+
# Spit out a warning if the archive hasn't an
|
95
|
+
# integrity checksum.
|
96
|
+
# [:tell_unsupported_check]
|
97
|
+
# Spit out a warning if the archive
|
98
|
+
# has an unsupported checksum type.
|
99
|
+
# [:concatenated]
|
100
|
+
# Decompress concatenated archives.
|
101
|
+
#
|
102
|
+
# [chunk (Block argument)]
|
103
|
+
# One piece of decompressed data.
|
104
|
+
#
|
105
|
+
# === Return value
|
106
|
+
#
|
107
|
+
# If a block was given, returns the number of bytes
|
108
|
+
# written. Otherwise, returns the decompressed data as a
|
109
|
+
# BINARY-encoded string.
|
110
|
+
#
|
111
|
+
# === Example
|
112
|
+
#
|
113
|
+
# data = File.open("archive.xz", "rb"){|f| f.read}
|
114
|
+
# io = StringIO.new(data)
|
115
|
+
#
|
116
|
+
# XZ.decompress_stream(io) #=> "I AM THE DATA"
|
117
|
+
# io.rewind
|
118
|
+
#
|
119
|
+
# str = ""
|
120
|
+
# XZ.decompress_stream(io, XZ::LibLZMA::UINT64_MAX, [:tell_no_check]){|c| str << c} #=> 13
|
121
|
+
# str #=> "I AM THE DATA"
|
122
|
+
#
|
123
|
+
# === Remarks
|
124
|
+
#
|
125
|
+
# The block form is *much* better on memory usage, because it
|
126
|
+
# doesn't have to load everything into RAM at once. If you don't
|
127
|
+
# know how big your data gets or if you want to decompress much
|
128
|
+
# data, use the block form. Of course you shouldn't store the data
|
129
|
+
# you read in RAM then as in the example above.
|
87
130
|
def decompress_stream(io, memory_limit = LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check], &block)
|
88
|
-
raise(ArgumentError, "Invalid memory limit set!") unless
|
89
|
-
|
90
|
-
|
131
|
+
raise(ArgumentError, "Invalid memory limit set!") unless memory_limit > 0 && memory_limit <= LibLZMA::UINT64_MAX
|
132
|
+
|
133
|
+
# bit-or all flags
|
134
|
+
allflags = flags.inject(0) do |val, flag|
|
135
|
+
flag = LibLZMA::LZMA_DECODE_FLAGS[flag] || raise(ArgumentError, "Unknown flag #{flag}!")
|
136
|
+
val | flag
|
91
137
|
end
|
92
138
|
|
93
139
|
stream = LZMAStream.new
|
94
|
-
res = LibLZMA.lzma_stream_decoder(
|
95
|
-
|
96
|
-
|
97
|
-
flags.inject(0){|val, flag| val | LibLZMA.const_get(:"LZMA_#{flag.to_s.upcase}")}
|
98
|
-
)
|
140
|
+
res = LibLZMA.lzma_stream_decoder(stream.pointer,
|
141
|
+
memory_limit,
|
142
|
+
allflags)
|
99
143
|
|
100
144
|
LZMAError.raise_if_necessary(res)
|
101
145
|
|
@@ -113,53 +157,74 @@ module XZ
|
|
113
157
|
end
|
114
158
|
alias decode_stream decompress_stream
|
115
159
|
|
116
|
-
#call-seq:
|
117
|
-
#
|
118
|
-
#
|
119
|
-
#
|
120
|
-
#
|
121
|
-
#
|
122
|
-
#Compresses a stream of data into XZ-compressed data.
|
123
|
-
|
124
|
-
#
|
125
|
-
#
|
126
|
-
#[
|
127
|
-
#
|
128
|
-
#
|
129
|
-
#
|
130
|
-
#
|
131
|
-
#
|
132
|
-
#
|
133
|
-
#
|
134
|
-
#
|
135
|
-
#[
|
136
|
-
#
|
137
|
-
#
|
138
|
-
#
|
139
|
-
|
140
|
-
#
|
141
|
-
#
|
142
|
-
|
143
|
-
#
|
144
|
-
#
|
145
|
-
#
|
146
|
-
#
|
147
|
-
#
|
148
|
-
#
|
149
|
-
#
|
150
|
-
|
151
|
-
#
|
152
|
-
#
|
153
|
-
#
|
154
|
-
#
|
155
|
-
#
|
160
|
+
# call-seq:
|
161
|
+
# compress_stream(io [, compression_level [, check [, extreme ] ] ] ) → a_string
|
162
|
+
# compress_stream(io [, compression_level [, check [, extreme ] ] ] ){|chunk| ... } → an_integer
|
163
|
+
# encode_stream(io [, compression_level [, check [, extreme ] ] ] ) → a_string
|
164
|
+
# encode_stream(io [, compression_level [, check [, extreme ] ] ] ){|chunk| ... } → an_integer
|
165
|
+
#
|
166
|
+
# Compresses a stream of data into XZ-compressed data.
|
167
|
+
#
|
168
|
+
# === Parameters
|
169
|
+
#
|
170
|
+
# [io]
|
171
|
+
# The IO to read the data from. Must be opened for
|
172
|
+
# reading.
|
173
|
+
#
|
174
|
+
# [compression_level (6)]
|
175
|
+
# Compression strength. Higher values indicate a
|
176
|
+
# smaller result, but longer compression time. Maximum
|
177
|
+
# is 9.
|
178
|
+
#
|
179
|
+
# [check (:crc64)]
|
180
|
+
# The checksum algorithm to use for verifying
|
181
|
+
# the data inside the archive. Possible values are:
|
182
|
+
# * :none
|
183
|
+
# * :crc32
|
184
|
+
# * :crc64
|
185
|
+
# * :sha256
|
186
|
+
#
|
187
|
+
# [extreme (false)]
|
188
|
+
# Tries to get the last bit out of the
|
189
|
+
# compression. This may succeed, but you can end
|
190
|
+
# up with *very* long computation times.
|
191
|
+
#
|
192
|
+
# [chunk (Block argument)]
|
193
|
+
# One piece of compressed data.
|
194
|
+
#
|
195
|
+
# === Return value
|
196
|
+
#
|
197
|
+
# If a block was given, returns the number of bytes
|
198
|
+
# written. Otherwise, returns the compressed data as a
|
199
|
+
# BINARY-encoded string.
|
200
|
+
#
|
201
|
+
# === Example
|
202
|
+
# data = File.read("file.txt")
|
203
|
+
# i = StringIO.new(data)
|
204
|
+
# XZ.compress_stream(i) #=> Some binary blob
|
205
|
+
#
|
206
|
+
# i.rewind
|
207
|
+
# str = ""
|
208
|
+
#
|
209
|
+
# XZ.compress_stream(i, 4, :sha256){|c| str << c} #=> 123
|
210
|
+
# str #=> Some binary blob
|
211
|
+
#
|
212
|
+
# === Remarks
|
213
|
+
#
|
214
|
+
# The block form is *much* better on memory usage, because it
|
215
|
+
# doesn't have to load everything into RAM at once. If you don't
|
216
|
+
# know how big your data gets or if you want to compress much
|
217
|
+
# data, use the block form. Of course you shouldn't store the data
|
218
|
+
# your read in RAM then as in the example above.
|
156
219
|
def compress_stream(io, compression_level = 6, check = :crc64, extreme = false, &block)
|
157
220
|
raise(ArgumentError, "Invalid compression level!") unless (0..9).include?(compression_level)
|
158
221
|
raise(ArgumentError, "Invalid checksum specified!") unless [:none, :crc32, :crc64, :sha256].include?(check)
|
159
222
|
|
223
|
+
compression_level |= LibLZMA::LZMA_PRESET_EXTREME if extreme
|
224
|
+
|
160
225
|
stream = LZMAStream.new
|
161
226
|
res = LibLZMA.lzma_easy_encoder(stream.pointer,
|
162
|
-
compression_level
|
227
|
+
compression_level,
|
163
228
|
LibLZMA::LZMA_CHECK[:"lzma_check_#{check}"])
|
164
229
|
|
165
230
|
LZMAError.raise_if_necessary(res)
|
@@ -178,20 +243,31 @@ module XZ
|
|
178
243
|
end
|
179
244
|
alias encode_stream compress_stream
|
180
245
|
|
181
|
-
#Compresses +in_file+ and writes the result to +out_file+.
|
182
|
-
|
183
|
-
#
|
184
|
-
#
|
185
|
-
#
|
186
|
-
#
|
187
|
-
|
188
|
-
#The
|
189
|
-
|
190
|
-
#
|
191
|
-
#
|
192
|
-
|
193
|
-
#
|
194
|
-
#
|
246
|
+
# Compresses +in_file+ and writes the result to +out_file+.
|
247
|
+
#
|
248
|
+
# === Parameters
|
249
|
+
#
|
250
|
+
# [in_file]
|
251
|
+
# The path to the file to read from.
|
252
|
+
# [out_file]
|
253
|
+
# The path of the file to write to. If it exists, it will be
|
254
|
+
# overwritten.
|
255
|
+
#
|
256
|
+
# For the other parameters, see the ::compress_stream method.
|
257
|
+
#
|
258
|
+
# === Return value
|
259
|
+
#
|
260
|
+
# The number of bytes written, i.e. the size of the archive.
|
261
|
+
#
|
262
|
+
# === Example
|
263
|
+
#
|
264
|
+
# XZ.compress("myfile.txt", "myfile.txt.xz")
|
265
|
+
# XZ.compress("myarchive.tar", "myarchive.tar.xz")
|
266
|
+
#
|
267
|
+
# === Remarks
|
268
|
+
#
|
269
|
+
# This method is safe to use with big files, because files are not
|
270
|
+
# loaded into memory completely at once.
|
195
271
|
def compress_file(in_file, out_file, compression_level = 6, check = :crc64, extreme = false)
|
196
272
|
File.open(in_file, "rb") do |i_file|
|
197
273
|
File.open(out_file, "wb") do |o_file|
|
@@ -202,56 +278,86 @@ module XZ
|
|
202
278
|
end
|
203
279
|
end
|
204
280
|
|
205
|
-
#Compresses arbitrary data using the XZ algorithm.
|
206
|
-
|
207
|
-
#
|
208
|
-
#
|
209
|
-
|
210
|
-
#
|
211
|
-
|
212
|
-
#
|
213
|
-
#
|
214
|
-
|
215
|
-
#
|
216
|
-
#
|
281
|
+
# Compresses arbitrary data using the XZ algorithm.
|
282
|
+
#
|
283
|
+
# === Parameters
|
284
|
+
#
|
285
|
+
# [str] The data to compress.
|
286
|
+
#
|
287
|
+
# For the other parameters, see the compress_stream method.
|
288
|
+
#
|
289
|
+
# === Return value
|
290
|
+
#
|
291
|
+
# The compressed data as a BINARY-encoded string.
|
292
|
+
#
|
293
|
+
# === Example
|
294
|
+
#
|
295
|
+
# data = "I love Ruby"
|
296
|
+
# comp = XZ.compress(data) #=> binary blob
|
297
|
+
#
|
298
|
+
# === Remarks
|
299
|
+
#
|
300
|
+
# Don't use this method for big amounts of data--you may run out
|
301
|
+
# of memory. Use compress_file or compress_stream instead.
|
217
302
|
def compress(str, compression_level = 6, check = :crc64, extreme = false)
|
218
303
|
raise(NotImplementedError, "StringIO isn't available!") unless defined? StringIO
|
219
304
|
s = StringIO.new(str)
|
220
305
|
compress_stream(s, compression_level, check, extreme)
|
221
306
|
end
|
222
307
|
|
223
|
-
#Decompresses data in XZ format.
|
224
|
-
|
225
|
-
#
|
226
|
-
#
|
227
|
-
|
228
|
-
#
|
229
|
-
|
230
|
-
#
|
231
|
-
#
|
232
|
-
|
233
|
-
#
|
234
|
-
#
|
308
|
+
# Decompresses data in XZ format.
|
309
|
+
#
|
310
|
+
# === Parameters
|
311
|
+
#
|
312
|
+
# [str] The data to decompress.
|
313
|
+
#
|
314
|
+
# For the other parameters, see the decompress_stream method.
|
315
|
+
#
|
316
|
+
# === Return value
|
317
|
+
#
|
318
|
+
# The decompressed data as a BINARY-encoded string.
|
319
|
+
#
|
320
|
+
# === Example
|
321
|
+
#
|
322
|
+
# comp = File.open("data.xz", "rb"){|f| f.read}
|
323
|
+
# data = XZ.decompress(comp) #=> "I love Ruby"
|
324
|
+
#
|
325
|
+
# === Remarks
|
326
|
+
#
|
327
|
+
# Don't use this method for big amounts of data--you may run out
|
328
|
+
# of memory. Use decompress_file or decompress_stream instead.
|
235
329
|
def decompress(str, memory_limit = LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check])
|
236
330
|
raise(NotImplementedError, "StringIO isn't available!") unless defined? StringIO
|
237
331
|
s = StringIO.new(str)
|
238
332
|
decompress_stream(s, memory_limit, flags)
|
239
333
|
end
|
240
334
|
|
241
|
-
#Decompresses +in_file+ and writes the result to +out_file+.
|
242
|
-
|
243
|
-
#
|
244
|
-
#
|
245
|
-
#
|
246
|
-
#
|
247
|
-
|
248
|
-
#The
|
249
|
-
|
250
|
-
#
|
251
|
-
#
|
252
|
-
|
253
|
-
#
|
254
|
-
#
|
335
|
+
# Decompresses +in_file+ and writes the result to +out_file+.
|
336
|
+
#
|
337
|
+
# ===Parameters
|
338
|
+
#
|
339
|
+
# [in_file]
|
340
|
+
# The path to the file to read from.
|
341
|
+
# [out_file]
|
342
|
+
# The path of the file to write to. If it exists, it will
|
343
|
+
# be overwritten.
|
344
|
+
#
|
345
|
+
# For the other parameters, see the decompress_stream method.
|
346
|
+
#
|
347
|
+
# === Return value
|
348
|
+
#
|
349
|
+
# The number of bytes written, i.e. the size of the uncompressed
|
350
|
+
# data.
|
351
|
+
#
|
352
|
+
# === Example
|
353
|
+
#
|
354
|
+
# XZ.decompres("myfile.txt.xz", "myfile.txt")
|
355
|
+
# XZ.decompress("myarchive.tar.xz", "myarchive.tar")
|
356
|
+
#
|
357
|
+
# === Remarks
|
358
|
+
#
|
359
|
+
# This method is safe to use with big files, because files are not
|
360
|
+
# loaded into memory completely at once.
|
255
361
|
def decompress_file(in_file, out_file, memory_limit = LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check])
|
256
362
|
File.open(in_file, "rb") do |i_file|
|
257
363
|
File.open(out_file, "wb") do |o_file|
|
@@ -264,24 +370,20 @@ module XZ
|
|
264
370
|
|
265
371
|
private
|
266
372
|
|
267
|
-
#This method returns the size of +str+ in bytes.
|
373
|
+
# This method returns the size of +str+ in bytes.
|
268
374
|
def binary_size(str)
|
269
|
-
#Believe it or not, but this is faster than str.bytes.to_a.size.
|
270
|
-
#I benchmarked it, and it is as twice as fast.
|
271
|
-
|
272
|
-
str.dup.force_encoding(Encoding::BINARY).size
|
273
|
-
else
|
274
|
-
str.bytes.to_a.size
|
275
|
-
end
|
375
|
+
# Believe it or not, but this is faster than str.bytes.to_a.size.
|
376
|
+
# I benchmarked it, and it is as twice as fast.
|
377
|
+
str.dup.force_encoding(Encoding::BINARY).size
|
276
378
|
end
|
277
379
|
|
278
|
-
#This method does the heavy work of (de-)compressing a stream. It
|
279
|
-
#an IO object to read data from (that means the IO must be
|
280
|
-
#for reading) and a XZ::LZMAStream object that is used to
|
281
|
-
#the data. Furthermore this method takes a block
|
282
|
-
#the (de-)compressed data in chunks one at a
|
283
|
-
#(de-)compressing of very large
|
284
|
-
#memory.
|
380
|
+
# This method does the heavy work of (de-)compressing a stream. It
|
381
|
+
# takes an IO object to read data from (that means the IO must be
|
382
|
+
# opened for reading) and a XZ::LZMAStream object that is used to
|
383
|
+
# (de-)compress the data. Furthermore this method takes a block
|
384
|
+
# which gets passed the (de-)compressed data in chunks one at a
|
385
|
+
# time--this is needed to allow (de-)compressing of very large
|
386
|
+
# files that can't be loaded fully into memory.
|
285
387
|
def lzma_code(io, stream)
|
286
388
|
input_buffer_p = FFI::MemoryPointer.new(CHUNK_SIZE)
|
287
389
|
output_buffer_p = FFI::MemoryPointer.new(CHUNK_SIZE)
|
@@ -289,24 +391,26 @@ module XZ
|
|
289
391
|
while str = io.read(CHUNK_SIZE)
|
290
392
|
input_buffer_p.write_string(str)
|
291
393
|
|
292
|
-
#Set the data for compressing
|
394
|
+
# Set the data for compressing
|
293
395
|
stream[:next_in] = input_buffer_p
|
294
396
|
stream[:avail_in] = binary_size(str)
|
295
397
|
|
296
|
-
#Now loop until we gathered all the data in
|
297
|
-
#amount of data, this may
|
298
|
-
#
|
299
|
-
#
|
300
|
-
#
|
301
|
-
#
|
302
|
-
#
|
303
|
-
#
|
398
|
+
# Now loop until we gathered all the data in
|
399
|
+
# stream[:next_out]. Depending on the amount of data, this may
|
400
|
+
# not fit into the buffer, meaning that we have to provide a
|
401
|
+
# pointer to a "new" buffer that liblzma can write into. Since
|
402
|
+
# liblzma already set stream[:avail_in] to 0 in the first
|
403
|
+
# iteration, the extra call to the lzma_code() function
|
404
|
+
# doesn't hurt (indeed the pipe_comp example from liblzma
|
405
|
+
# handles it this way too). Sometimes it happens that the
|
406
|
+
# compressed data is bigger than the original (notably when
|
407
|
+
# the amount of data to compress is small).
|
304
408
|
loop do
|
305
|
-
#Prepare for getting the compressed_data
|
409
|
+
# Prepare for getting the compressed_data
|
306
410
|
stream[:next_out] = output_buffer_p
|
307
411
|
stream[:avail_out] = CHUNK_SIZE
|
308
412
|
|
309
|
-
#Compress the data
|
413
|
+
# Compress the data
|
310
414
|
res = if io.eof?
|
311
415
|
LibLZMA.lzma_code(stream.pointer, LibLZMA::LZMA_ACTION[:lzma_finish])
|
312
416
|
else
|
@@ -314,26 +418,27 @@ module XZ
|
|
314
418
|
end
|
315
419
|
check_lzma_code_retval(res)
|
316
420
|
|
317
|
-
#Write the compressed data
|
421
|
+
# Write the compressed data
|
318
422
|
data = output_buffer_p.read_string(CHUNK_SIZE - stream[:avail_out])
|
319
423
|
yield(data)
|
320
424
|
|
321
|
-
#If the buffer is completely filled, it's likely that there
|
322
|
-
#more data liblzma wants to hand to us. Start a new
|
323
|
-
#but don't provide new input data.
|
425
|
+
# If the buffer is completely filled, it's likely that there
|
426
|
+
# is more data liblzma wants to hand to us. Start a new
|
427
|
+
# iteration, but don't provide new input data.
|
324
428
|
break unless stream[:avail_out] == 0
|
325
429
|
end #loop
|
326
430
|
end #while
|
327
431
|
end #lzma_code
|
328
432
|
|
329
|
-
#Checks for errors and warnings that can be derived from the
|
330
|
-
#value of the lzma_code() function and shows them if
|
433
|
+
# Checks for errors and warnings that can be derived from the
|
434
|
+
# return value of the lzma_code() function and shows them if
|
435
|
+
# necessary.
|
331
436
|
def check_lzma_code_retval(code)
|
332
437
|
e = LibLZMA::LZMA_RET
|
333
438
|
case code
|
334
|
-
when e[:lzma_no_check] then warn("Couldn't verify archive integrity--archive has
|
439
|
+
when e[:lzma_no_check] then warn("Couldn't verify archive integrity--archive has no integrity checksum.")
|
335
440
|
when e[:lzma_unsupported_check] then warn("Couldn't verify archive integrity--archive has an unsupported integrity checksum.")
|
336
|
-
when e[:lzma_get_check] then nil #This isn't useful for us. It indicates that the checksum type is now known.
|
441
|
+
when e[:lzma_get_check] then nil # This isn't useful for us. It indicates that the checksum type is now known.
|
337
442
|
else
|
338
443
|
LZMAError.raise_if_necessary(code)
|
339
444
|
end
|
@@ -343,6 +448,7 @@ module XZ
|
|
343
448
|
|
344
449
|
end
|
345
450
|
|
451
|
+
require_relative "xz/version"
|
346
452
|
require_relative "xz/lib_lzma"
|
347
453
|
require_relative "xz/stream"
|
348
454
|
require_relative "xz/stream_writer"
|