ruby-xz 0.2.1 → 0.2.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/AUTHORS +8 -0
- data/COPYING +1 -1
- data/HISTORY.rdoc +24 -4
- data/{README.rdoc → README.md} +61 -33
- data/lib/xz.rb +290 -184
- data/lib/xz/lib_lzma.rb +28 -18
- data/lib/xz/stream.rb +24 -22
- data/lib/xz/stream_reader.rb +328 -122
- data/lib/xz/stream_writer.rb +288 -97
- data/lib/xz/version.rb +33 -0
- metadata +36 -46
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 28c451853871a094353485f28db08d64c5a5effa
|
4
|
+
data.tar.gz: 93ef5cba40e6a6330364206caf86dbcca6f95a7e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 75552a3ee976a8441ff397953bd608a8e9e6e5508b9c7cae73eb05278f509475c00fcea7ade557f758520c5b06b42711d96c162e9e5f2f5cc7ebed05e810a99f
|
7
|
+
data.tar.gz: 11b58af7569d56fe0c7486945a9e5ef87bfb0bf9579d29a82c3f7f21f8aa9ae2d03e118927bdc295585e857b82d5d536ddb9012393646591dc4a58585462587e
|
data/AUTHORS
ADDED
data/COPYING
CHANGED
data/HISTORY.rdoc
CHANGED
@@ -1,16 +1,36 @@
|
|
1
|
-
|
1
|
+
= Version history
|
2
|
+
|
3
|
+
== 0.2.2 (unreleased)
|
4
|
+
|
5
|
+
* *Add* XZ.disable_deprecation_notes
|
6
|
+
* *Deprecate* use of XZ::StreamReader.open with an IO argument
|
7
|
+
* *Deprecate* use of XZ::StreamReader.new with a filename argument
|
8
|
+
* *Deprecate* use of XZ::StreamWriter.open with an IO argument
|
9
|
+
* *Deprecate* use of XZ::StreamWriter.new with a filename argument
|
10
|
+
* *Deprecate* nonautomatic IO close in XZ::StreamReader#close
|
11
|
+
* *Deprecate* nonautomatic IO close in XZ::StreamWriter#close
|
12
|
+
* *Fix* incompatibility with Resolv.getaddress() in Ruby 2.2 (Ticket #13
|
13
|
+
by Ken Simon)
|
14
|
+
* Goal of these deprecations is to sync the API with Ruby’s own
|
15
|
+
Zlib::GzipWriter and Zlib::GzipReader mostly.
|
16
|
+
* Add required versions to gemspec.
|
17
|
+
* Comment format cleanup, results in better docs.
|
18
|
+
* Internal code cleanup
|
19
|
+
* Add more tests.
|
20
|
+
|
21
|
+
== 0.2.1 (2014-02-08)
|
2
22
|
|
3
23
|
* Build the gem properly on Ruby 2.0+ (PR #8 by Nana Sakisaka (saki7))
|
4
24
|
* Release the GIL when interfacing with liblzma (PR #7 by Lars Christensen (larsch))
|
5
25
|
|
6
|
-
== 0.2.0
|
26
|
+
== 0.2.0 (2013-06-23)
|
7
27
|
|
8
28
|
* Fix #6 (errors on JRuby) by Ben Nagy
|
9
29
|
* <b>Remove 1.8 compatibility</b>
|
10
30
|
|
11
|
-
== 0.1.0
|
31
|
+
== 0.1.0 (2013-02-17)
|
12
32
|
|
13
33
|
* <b>Add XZ::StreamReader and XZ::StreamWriter for io-like behaviour.</b>
|
14
34
|
* New dependency on the +io-like+ gem.
|
15
35
|
* <b>Add Ruby 1.8 compatibility.</b> Thanks to Christoph Plank.
|
16
|
-
* We now have proper unit tests.
|
36
|
+
* We now have proper unit tests.
|
data/{README.rdoc → README.md}
RENAMED
@@ -1,11 +1,12 @@
|
|
1
|
-
|
1
|
+
ruby-xz
|
2
|
+
=======
|
2
3
|
|
3
|
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
4
|
+
**ruby-xz** is a basic binding to the famous [liblzma library][1],
|
5
|
+
best known for the extreme compression-ratio it's native *XZ* format
|
6
|
+
achieves. ruby-xz gives you the possibility of creating and extracting
|
7
|
+
XZ archives on any platform where liblzma is installed. No compilation
|
8
|
+
is needed, because ruby-xz is written ontop of
|
9
|
+
[ffi][2].
|
9
10
|
|
10
11
|
ruby-xz supports both "intuitive" (de)compression by providing methods to
|
11
12
|
directly operate on strings and files, but also allows you to operate
|
@@ -14,59 +15,82 @@ of that, ruby-xz offers an advanced interface that allows you to treat
|
|
14
15
|
XZ-compressed data as IO streams, both for reading and for writing. See the
|
15
16
|
XZ::StreamReader and XZ::StreamWriter classes for more information on this.
|
16
17
|
|
17
|
-
|
18
|
+
Installation
|
19
|
+
------------
|
18
20
|
|
19
21
|
Install it the way you install all your gems.
|
20
22
|
|
21
|
-
|
23
|
+
```
|
24
|
+
$ gem install ruby-xz
|
25
|
+
```
|
22
26
|
|
23
|
-
|
27
|
+
Alternatively, you can clone the repository and build the most recent
|
28
|
+
code yourself:
|
24
29
|
|
25
|
-
|
30
|
+
```
|
31
|
+
$ git clone git://github.com/Quintus/ruby-xz.git
|
32
|
+
$ cd ruby-xz
|
33
|
+
$ rake gem
|
34
|
+
$ gem install pkg/ruby-xz-*.gem
|
35
|
+
```
|
36
|
+
|
37
|
+
Usage
|
38
|
+
-----
|
26
39
|
|
27
40
|
The documentation of the XZ module is well and you should be able to find
|
28
41
|
everything you need to use ruby-xz. As said, it's not big, but powerful:
|
29
42
|
You can create and extract whole archive files, compress or decompress
|
30
43
|
streams of data or just plain strings.
|
31
44
|
|
32
|
-
You can read the documentation on your local gemserver, or online
|
33
|
-
http://quintus.github.io/ruby-xz/.
|
45
|
+
You can read the documentation on your local gemserver, or browse it [online][3].
|
34
46
|
|
35
|
-
|
47
|
+
### First step ###
|
36
48
|
|
37
49
|
You have to require ruby-xz. Note the file you have to require is named
|
38
50
|
"xz.rb", so do
|
39
51
|
|
40
|
-
|
41
|
-
|
52
|
+
``` ruby
|
53
|
+
require "xz"
|
54
|
+
```
|
55
|
+
|
42
56
|
to get it.
|
43
57
|
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
58
|
+
### Examples ###
|
59
|
+
|
60
|
+
``` ruby
|
61
|
+
# Compress a TAR archive
|
62
|
+
XZ.compress_file("myfile.tar", "myfile.tar.xz")
|
63
|
+
# Decompress it
|
64
|
+
XZ.decompress_file("myfile.tar.xz", "myfile.tar")
|
65
|
+
|
66
|
+
# Compress everything you get from a socket (note that there HAS to be a EOF
|
67
|
+
# sometime, otherwise this will run infinitely)
|
68
|
+
XZ.compress_stream(socket){|chunk| opened_file.write(chunk)}
|
69
|
+
|
70
|
+
# Compress a string
|
71
|
+
comp = XZ.compress("Mydata")
|
72
|
+
# Decompress it
|
73
|
+
data = XZ.decompress(comp)
|
74
|
+
```
|
59
75
|
|
60
76
|
Have a look at the XZ module's documentation for an in-depth description of
|
61
77
|
what is possible.
|
62
78
|
|
63
|
-
|
79
|
+
Links
|
80
|
+
-----
|
81
|
+
|
82
|
+
* Code repository: https://github.com/Quintus/ruby-xz
|
83
|
+
* Issue tracker: https://github.com/Quintus/ruby-xz/issues
|
84
|
+
* Online documentation: http://quintus.github.io/ruby-xz
|
85
|
+
|
86
|
+
License
|
87
|
+
-------
|
64
88
|
|
65
89
|
(The MIT License)
|
66
90
|
|
67
91
|
Basic liblzma-bindings for Ruby.
|
68
92
|
|
69
|
-
Copyright © 2011-
|
93
|
+
Copyright © 2011-2015 Marvin Gülker et al.
|
70
94
|
|
71
95
|
See AUTHORS for the full list of contributors.
|
72
96
|
|
@@ -87,3 +111,7 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
87
111
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
88
112
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
89
113
|
THE SOFTWARE.
|
114
|
+
|
115
|
+
[1]: http://tukaani.org/xz/
|
116
|
+
[2]: https://github.com/ffi/ffi
|
117
|
+
[3]: http://quintus.github.io/ruby-xz
|
data/lib/xz.rb
CHANGED
@@ -1,9 +1,10 @@
|
|
1
1
|
# -*- coding: utf-8 -*-
|
2
|
+
#--
|
2
3
|
# (The MIT License)
|
3
4
|
#
|
4
5
|
# Basic liblzma-bindings for Ruby.
|
5
6
|
#
|
6
|
-
# Copyright © 2011,2012 Marvin Gülker
|
7
|
+
# Copyright © 2011,2012,2015 Marvin Gülker
|
7
8
|
# Copyright © 2011 Christoph Plank
|
8
9
|
#
|
9
10
|
# Permission is hereby granted, free of charge, to any person obtaining a
|
@@ -23,79 +24,122 @@
|
|
23
24
|
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
24
25
|
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
25
26
|
# THE SOFTWARE.
|
27
|
+
#++
|
26
28
|
|
27
29
|
require "pathname"
|
28
30
|
require "ffi"
|
29
31
|
require 'stringio'
|
30
32
|
require "io/like"
|
31
33
|
|
32
|
-
#The namespace and main module of this library. Each method of this
|
33
|
-
#may raise exceptions of class XZ::LZMAError, which is not
|
34
|
-
#methods' documentations anymore.
|
34
|
+
# The namespace and main module of this library. Each method of this
|
35
|
+
# module may raise exceptions of class XZ::LZMAError, which is not
|
36
|
+
# named in the methods' documentations anymore.
|
35
37
|
#
|
36
|
-
#All strings you receive from any method defined in this module
|
37
|
-
#
|
38
|
-
#
|
39
|
-
#
|
40
|
-
#ruby-xz can’t handle this as compiled strings don’t come with
|
41
|
-
#information.
|
38
|
+
# All strings you receive from any method defined in this module and
|
39
|
+
# the classes defined in it are encoded in BINARY, so you may have to
|
40
|
+
# call #force_encoding on them to tag them with the correct encoding
|
41
|
+
# (assuming you _know_ what their correct encoding should be).
|
42
|
+
# ruby-xz can’t handle this as compiled strings don’t come with
|
43
|
+
# encoding information.
|
42
44
|
module XZ
|
43
|
-
#The version of this library.
|
44
|
-
VERSION = "0.2.1"
|
45
45
|
|
46
|
-
#Number of bytes read in one chunk.
|
46
|
+
# Number of bytes read in one chunk.
|
47
47
|
CHUNK_SIZE = 4096
|
48
48
|
|
49
49
|
class << self
|
50
50
|
|
51
|
-
#
|
52
|
-
#
|
53
|
-
#
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
#
|
63
|
-
#
|
64
|
-
#
|
65
|
-
#
|
66
|
-
#
|
67
|
-
#
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
#
|
73
|
-
|
74
|
-
#
|
75
|
-
#
|
76
|
-
#
|
77
|
-
#
|
78
|
-
#
|
79
|
-
#
|
80
|
-
#
|
81
|
-
|
82
|
-
#
|
83
|
-
#
|
84
|
-
#
|
85
|
-
#
|
86
|
-
#
|
51
|
+
# :nodoc:
|
52
|
+
#
|
53
|
+
# Output a deprecation notice.
|
54
|
+
def deprecate(msg)
|
55
|
+
@disable_deprecation_notices ||= false
|
56
|
+
|
57
|
+
unless @disable_deprecation_notices
|
58
|
+
$stderr.puts("DEPRECATION NOTICE: #{msg}\n#{caller.drop(1).join("\n\t")}")
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
62
|
+
# Force ruby-xz to be silent about deprecations. Using this is
|
63
|
+
# discouraged so that you are aware of upcoming changes to the
|
64
|
+
# API. However, if your standard error stream is closed,
|
65
|
+
# outputting the deprecation notices might result in an exception,
|
66
|
+
# so this method allows you to surpress these notices. Ensure you
|
67
|
+
# read the HISTORY.rdoc file carefully instead.
|
68
|
+
def disable_deprecation_notices=(bool)
|
69
|
+
@disable_deprecation_notices = bool
|
70
|
+
end
|
71
|
+
|
72
|
+
# call-seq:
|
73
|
+
# decompress_stream(io [, memory_limit [, flags ] ] ) → a_string
|
74
|
+
# decompress_stream(io [, memory_limit [, flags ] ] ){|chunk| ... } → an_integer
|
75
|
+
# decode_stream(io [, memory_limit [, flags ] ] ) → a_string
|
76
|
+
# decode_stream(io [, memory_limit [, flags ] ] ){|chunk| ... } → an_integer
|
77
|
+
#
|
78
|
+
# Decompresses a stream containing XZ-compressed data.
|
79
|
+
#
|
80
|
+
# === Parameters
|
81
|
+
#
|
82
|
+
# [io]
|
83
|
+
# The IO to read from. It must be opened for reading.
|
84
|
+
#
|
85
|
+
# [memory_limit (+UINT64_MAX+)]
|
86
|
+
# If not XZ::LibLZMA::UINT64_MAX, makes liblzma
|
87
|
+
# use no more memory than +memory_limit+ bytes.
|
88
|
+
#
|
89
|
+
# [flags (<tt>[:tell_unsupported_check]</tt>)]
|
90
|
+
# Additional flags
|
91
|
+
# passed to liblzma (an array). Possible flags are:
|
92
|
+
#
|
93
|
+
# [:tell_no_check]
|
94
|
+
# Spit out a warning if the archive hasn't an
|
95
|
+
# integrity checksum.
|
96
|
+
# [:tell_unsupported_check]
|
97
|
+
# Spit out a warning if the archive
|
98
|
+
# has an unsupported checksum type.
|
99
|
+
# [:concatenated]
|
100
|
+
# Decompress concatenated archives.
|
101
|
+
#
|
102
|
+
# [chunk (Block argument)]
|
103
|
+
# One piece of decompressed data.
|
104
|
+
#
|
105
|
+
# === Return value
|
106
|
+
#
|
107
|
+
# If a block was given, returns the number of bytes
|
108
|
+
# written. Otherwise, returns the decompressed data as a
|
109
|
+
# BINARY-encoded string.
|
110
|
+
#
|
111
|
+
# === Example
|
112
|
+
#
|
113
|
+
# data = File.open("archive.xz", "rb"){|f| f.read}
|
114
|
+
# io = StringIO.new(data)
|
115
|
+
#
|
116
|
+
# XZ.decompress_stream(io) #=> "I AM THE DATA"
|
117
|
+
# io.rewind
|
118
|
+
#
|
119
|
+
# str = ""
|
120
|
+
# XZ.decompress_stream(io, XZ::LibLZMA::UINT64_MAX, [:tell_no_check]){|c| str << c} #=> 13
|
121
|
+
# str #=> "I AM THE DATA"
|
122
|
+
#
|
123
|
+
# === Remarks
|
124
|
+
#
|
125
|
+
# The block form is *much* better on memory usage, because it
|
126
|
+
# doesn't have to load everything into RAM at once. If you don't
|
127
|
+
# know how big your data gets or if you want to decompress much
|
128
|
+
# data, use the block form. Of course you shouldn't store the data
|
129
|
+
# you read in RAM then as in the example above.
|
87
130
|
def decompress_stream(io, memory_limit = LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check], &block)
|
88
|
-
raise(ArgumentError, "Invalid memory limit set!") unless
|
89
|
-
|
90
|
-
|
131
|
+
raise(ArgumentError, "Invalid memory limit set!") unless memory_limit > 0 && memory_limit <= LibLZMA::UINT64_MAX
|
132
|
+
|
133
|
+
# bit-or all flags
|
134
|
+
allflags = flags.inject(0) do |val, flag|
|
135
|
+
flag = LibLZMA::LZMA_DECODE_FLAGS[flag] || raise(ArgumentError, "Unknown flag #{flag}!")
|
136
|
+
val | flag
|
91
137
|
end
|
92
138
|
|
93
139
|
stream = LZMAStream.new
|
94
|
-
res = LibLZMA.lzma_stream_decoder(
|
95
|
-
|
96
|
-
|
97
|
-
flags.inject(0){|val, flag| val | LibLZMA.const_get(:"LZMA_#{flag.to_s.upcase}")}
|
98
|
-
)
|
140
|
+
res = LibLZMA.lzma_stream_decoder(stream.pointer,
|
141
|
+
memory_limit,
|
142
|
+
allflags)
|
99
143
|
|
100
144
|
LZMAError.raise_if_necessary(res)
|
101
145
|
|
@@ -113,53 +157,74 @@ module XZ
|
|
113
157
|
end
|
114
158
|
alias decode_stream decompress_stream
|
115
159
|
|
116
|
-
#call-seq:
|
117
|
-
#
|
118
|
-
#
|
119
|
-
#
|
120
|
-
#
|
121
|
-
#
|
122
|
-
#Compresses a stream of data into XZ-compressed data.
|
123
|
-
|
124
|
-
#
|
125
|
-
#
|
126
|
-
#[
|
127
|
-
#
|
128
|
-
#
|
129
|
-
#
|
130
|
-
#
|
131
|
-
#
|
132
|
-
#
|
133
|
-
#
|
134
|
-
#
|
135
|
-
#[
|
136
|
-
#
|
137
|
-
#
|
138
|
-
#
|
139
|
-
|
140
|
-
#
|
141
|
-
#
|
142
|
-
|
143
|
-
#
|
144
|
-
#
|
145
|
-
#
|
146
|
-
#
|
147
|
-
#
|
148
|
-
#
|
149
|
-
#
|
150
|
-
|
151
|
-
#
|
152
|
-
#
|
153
|
-
#
|
154
|
-
#
|
155
|
-
#
|
160
|
+
# call-seq:
|
161
|
+
# compress_stream(io [, compression_level [, check [, extreme ] ] ] ) → a_string
|
162
|
+
# compress_stream(io [, compression_level [, check [, extreme ] ] ] ){|chunk| ... } → an_integer
|
163
|
+
# encode_stream(io [, compression_level [, check [, extreme ] ] ] ) → a_string
|
164
|
+
# encode_stream(io [, compression_level [, check [, extreme ] ] ] ){|chunk| ... } → an_integer
|
165
|
+
#
|
166
|
+
# Compresses a stream of data into XZ-compressed data.
|
167
|
+
#
|
168
|
+
# === Parameters
|
169
|
+
#
|
170
|
+
# [io]
|
171
|
+
# The IO to read the data from. Must be opened for
|
172
|
+
# reading.
|
173
|
+
#
|
174
|
+
# [compression_level (6)]
|
175
|
+
# Compression strength. Higher values indicate a
|
176
|
+
# smaller result, but longer compression time. Maximum
|
177
|
+
# is 9.
|
178
|
+
#
|
179
|
+
# [check (:crc64)]
|
180
|
+
# The checksum algorithm to use for verifying
|
181
|
+
# the data inside the archive. Possible values are:
|
182
|
+
# * :none
|
183
|
+
# * :crc32
|
184
|
+
# * :crc64
|
185
|
+
# * :sha256
|
186
|
+
#
|
187
|
+
# [extreme (false)]
|
188
|
+
# Tries to get the last bit out of the
|
189
|
+
# compression. This may succeed, but you can end
|
190
|
+
# up with *very* long computation times.
|
191
|
+
#
|
192
|
+
# [chunk (Block argument)]
|
193
|
+
# One piece of compressed data.
|
194
|
+
#
|
195
|
+
# === Return value
|
196
|
+
#
|
197
|
+
# If a block was given, returns the number of bytes
|
198
|
+
# written. Otherwise, returns the compressed data as a
|
199
|
+
# BINARY-encoded string.
|
200
|
+
#
|
201
|
+
# === Example
|
202
|
+
# data = File.read("file.txt")
|
203
|
+
# i = StringIO.new(data)
|
204
|
+
# XZ.compress_stream(i) #=> Some binary blob
|
205
|
+
#
|
206
|
+
# i.rewind
|
207
|
+
# str = ""
|
208
|
+
#
|
209
|
+
# XZ.compress_stream(i, 4, :sha256){|c| str << c} #=> 123
|
210
|
+
# str #=> Some binary blob
|
211
|
+
#
|
212
|
+
# === Remarks
|
213
|
+
#
|
214
|
+
# The block form is *much* better on memory usage, because it
|
215
|
+
# doesn't have to load everything into RAM at once. If you don't
|
216
|
+
# know how big your data gets or if you want to compress much
|
217
|
+
# data, use the block form. Of course you shouldn't store the data
|
218
|
+
# your read in RAM then as in the example above.
|
156
219
|
def compress_stream(io, compression_level = 6, check = :crc64, extreme = false, &block)
|
157
220
|
raise(ArgumentError, "Invalid compression level!") unless (0..9).include?(compression_level)
|
158
221
|
raise(ArgumentError, "Invalid checksum specified!") unless [:none, :crc32, :crc64, :sha256].include?(check)
|
159
222
|
|
223
|
+
compression_level |= LibLZMA::LZMA_PRESET_EXTREME if extreme
|
224
|
+
|
160
225
|
stream = LZMAStream.new
|
161
226
|
res = LibLZMA.lzma_easy_encoder(stream.pointer,
|
162
|
-
compression_level
|
227
|
+
compression_level,
|
163
228
|
LibLZMA::LZMA_CHECK[:"lzma_check_#{check}"])
|
164
229
|
|
165
230
|
LZMAError.raise_if_necessary(res)
|
@@ -178,20 +243,31 @@ module XZ
|
|
178
243
|
end
|
179
244
|
alias encode_stream compress_stream
|
180
245
|
|
181
|
-
#Compresses +in_file+ and writes the result to +out_file+.
|
182
|
-
|
183
|
-
#
|
184
|
-
#
|
185
|
-
#
|
186
|
-
#
|
187
|
-
|
188
|
-
#The
|
189
|
-
|
190
|
-
#
|
191
|
-
#
|
192
|
-
|
193
|
-
#
|
194
|
-
#
|
246
|
+
# Compresses +in_file+ and writes the result to +out_file+.
|
247
|
+
#
|
248
|
+
# === Parameters
|
249
|
+
#
|
250
|
+
# [in_file]
|
251
|
+
# The path to the file to read from.
|
252
|
+
# [out_file]
|
253
|
+
# The path of the file to write to. If it exists, it will be
|
254
|
+
# overwritten.
|
255
|
+
#
|
256
|
+
# For the other parameters, see the ::compress_stream method.
|
257
|
+
#
|
258
|
+
# === Return value
|
259
|
+
#
|
260
|
+
# The number of bytes written, i.e. the size of the archive.
|
261
|
+
#
|
262
|
+
# === Example
|
263
|
+
#
|
264
|
+
# XZ.compress("myfile.txt", "myfile.txt.xz")
|
265
|
+
# XZ.compress("myarchive.tar", "myarchive.tar.xz")
|
266
|
+
#
|
267
|
+
# === Remarks
|
268
|
+
#
|
269
|
+
# This method is safe to use with big files, because files are not
|
270
|
+
# loaded into memory completely at once.
|
195
271
|
def compress_file(in_file, out_file, compression_level = 6, check = :crc64, extreme = false)
|
196
272
|
File.open(in_file, "rb") do |i_file|
|
197
273
|
File.open(out_file, "wb") do |o_file|
|
@@ -202,56 +278,86 @@ module XZ
|
|
202
278
|
end
|
203
279
|
end
|
204
280
|
|
205
|
-
#Compresses arbitrary data using the XZ algorithm.
|
206
|
-
|
207
|
-
#
|
208
|
-
#
|
209
|
-
|
210
|
-
#
|
211
|
-
|
212
|
-
#
|
213
|
-
#
|
214
|
-
|
215
|
-
#
|
216
|
-
#
|
281
|
+
# Compresses arbitrary data using the XZ algorithm.
|
282
|
+
#
|
283
|
+
# === Parameters
|
284
|
+
#
|
285
|
+
# [str] The data to compress.
|
286
|
+
#
|
287
|
+
# For the other parameters, see the compress_stream method.
|
288
|
+
#
|
289
|
+
# === Return value
|
290
|
+
#
|
291
|
+
# The compressed data as a BINARY-encoded string.
|
292
|
+
#
|
293
|
+
# === Example
|
294
|
+
#
|
295
|
+
# data = "I love Ruby"
|
296
|
+
# comp = XZ.compress(data) #=> binary blob
|
297
|
+
#
|
298
|
+
# === Remarks
|
299
|
+
#
|
300
|
+
# Don't use this method for big amounts of data--you may run out
|
301
|
+
# of memory. Use compress_file or compress_stream instead.
|
217
302
|
def compress(str, compression_level = 6, check = :crc64, extreme = false)
|
218
303
|
raise(NotImplementedError, "StringIO isn't available!") unless defined? StringIO
|
219
304
|
s = StringIO.new(str)
|
220
305
|
compress_stream(s, compression_level, check, extreme)
|
221
306
|
end
|
222
307
|
|
223
|
-
#Decompresses data in XZ format.
|
224
|
-
|
225
|
-
#
|
226
|
-
#
|
227
|
-
|
228
|
-
#
|
229
|
-
|
230
|
-
#
|
231
|
-
#
|
232
|
-
|
233
|
-
#
|
234
|
-
#
|
308
|
+
# Decompresses data in XZ format.
|
309
|
+
#
|
310
|
+
# === Parameters
|
311
|
+
#
|
312
|
+
# [str] The data to decompress.
|
313
|
+
#
|
314
|
+
# For the other parameters, see the decompress_stream method.
|
315
|
+
#
|
316
|
+
# === Return value
|
317
|
+
#
|
318
|
+
# The decompressed data as a BINARY-encoded string.
|
319
|
+
#
|
320
|
+
# === Example
|
321
|
+
#
|
322
|
+
# comp = File.open("data.xz", "rb"){|f| f.read}
|
323
|
+
# data = XZ.decompress(comp) #=> "I love Ruby"
|
324
|
+
#
|
325
|
+
# === Remarks
|
326
|
+
#
|
327
|
+
# Don't use this method for big amounts of data--you may run out
|
328
|
+
# of memory. Use decompress_file or decompress_stream instead.
|
235
329
|
def decompress(str, memory_limit = LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check])
|
236
330
|
raise(NotImplementedError, "StringIO isn't available!") unless defined? StringIO
|
237
331
|
s = StringIO.new(str)
|
238
332
|
decompress_stream(s, memory_limit, flags)
|
239
333
|
end
|
240
334
|
|
241
|
-
#Decompresses +in_file+ and writes the result to +out_file+.
|
242
|
-
|
243
|
-
#
|
244
|
-
#
|
245
|
-
#
|
246
|
-
#
|
247
|
-
|
248
|
-
#The
|
249
|
-
|
250
|
-
#
|
251
|
-
#
|
252
|
-
|
253
|
-
#
|
254
|
-
#
|
335
|
+
# Decompresses +in_file+ and writes the result to +out_file+.
|
336
|
+
#
|
337
|
+
# ===Parameters
|
338
|
+
#
|
339
|
+
# [in_file]
|
340
|
+
# The path to the file to read from.
|
341
|
+
# [out_file]
|
342
|
+
# The path of the file to write to. If it exists, it will
|
343
|
+
# be overwritten.
|
344
|
+
#
|
345
|
+
# For the other parameters, see the decompress_stream method.
|
346
|
+
#
|
347
|
+
# === Return value
|
348
|
+
#
|
349
|
+
# The number of bytes written, i.e. the size of the uncompressed
|
350
|
+
# data.
|
351
|
+
#
|
352
|
+
# === Example
|
353
|
+
#
|
354
|
+
# XZ.decompres("myfile.txt.xz", "myfile.txt")
|
355
|
+
# XZ.decompress("myarchive.tar.xz", "myarchive.tar")
|
356
|
+
#
|
357
|
+
# === Remarks
|
358
|
+
#
|
359
|
+
# This method is safe to use with big files, because files are not
|
360
|
+
# loaded into memory completely at once.
|
255
361
|
def decompress_file(in_file, out_file, memory_limit = LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check])
|
256
362
|
File.open(in_file, "rb") do |i_file|
|
257
363
|
File.open(out_file, "wb") do |o_file|
|
@@ -264,24 +370,20 @@ module XZ
|
|
264
370
|
|
265
371
|
private
|
266
372
|
|
267
|
-
#This method returns the size of +str+ in bytes.
|
373
|
+
# This method returns the size of +str+ in bytes.
|
268
374
|
def binary_size(str)
|
269
|
-
#Believe it or not, but this is faster than str.bytes.to_a.size.
|
270
|
-
#I benchmarked it, and it is as twice as fast.
|
271
|
-
|
272
|
-
str.dup.force_encoding(Encoding::BINARY).size
|
273
|
-
else
|
274
|
-
str.bytes.to_a.size
|
275
|
-
end
|
375
|
+
# Believe it or not, but this is faster than str.bytes.to_a.size.
|
376
|
+
# I benchmarked it, and it is as twice as fast.
|
377
|
+
str.dup.force_encoding(Encoding::BINARY).size
|
276
378
|
end
|
277
379
|
|
278
|
-
#This method does the heavy work of (de-)compressing a stream. It
|
279
|
-
#an IO object to read data from (that means the IO must be
|
280
|
-
#for reading) and a XZ::LZMAStream object that is used to
|
281
|
-
#the data. Furthermore this method takes a block
|
282
|
-
#the (de-)compressed data in chunks one at a
|
283
|
-
#(de-)compressing of very large
|
284
|
-
#memory.
|
380
|
+
# This method does the heavy work of (de-)compressing a stream. It
|
381
|
+
# takes an IO object to read data from (that means the IO must be
|
382
|
+
# opened for reading) and a XZ::LZMAStream object that is used to
|
383
|
+
# (de-)compress the data. Furthermore this method takes a block
|
384
|
+
# which gets passed the (de-)compressed data in chunks one at a
|
385
|
+
# time--this is needed to allow (de-)compressing of very large
|
386
|
+
# files that can't be loaded fully into memory.
|
285
387
|
def lzma_code(io, stream)
|
286
388
|
input_buffer_p = FFI::MemoryPointer.new(CHUNK_SIZE)
|
287
389
|
output_buffer_p = FFI::MemoryPointer.new(CHUNK_SIZE)
|
@@ -289,24 +391,26 @@ module XZ
|
|
289
391
|
while str = io.read(CHUNK_SIZE)
|
290
392
|
input_buffer_p.write_string(str)
|
291
393
|
|
292
|
-
#Set the data for compressing
|
394
|
+
# Set the data for compressing
|
293
395
|
stream[:next_in] = input_buffer_p
|
294
396
|
stream[:avail_in] = binary_size(str)
|
295
397
|
|
296
|
-
#Now loop until we gathered all the data in
|
297
|
-
#amount of data, this may
|
298
|
-
#
|
299
|
-
#
|
300
|
-
#
|
301
|
-
#
|
302
|
-
#
|
303
|
-
#
|
398
|
+
# Now loop until we gathered all the data in
|
399
|
+
# stream[:next_out]. Depending on the amount of data, this may
|
400
|
+
# not fit into the buffer, meaning that we have to provide a
|
401
|
+
# pointer to a "new" buffer that liblzma can write into. Since
|
402
|
+
# liblzma already set stream[:avail_in] to 0 in the first
|
403
|
+
# iteration, the extra call to the lzma_code() function
|
404
|
+
# doesn't hurt (indeed the pipe_comp example from liblzma
|
405
|
+
# handles it this way too). Sometimes it happens that the
|
406
|
+
# compressed data is bigger than the original (notably when
|
407
|
+
# the amount of data to compress is small).
|
304
408
|
loop do
|
305
|
-
#Prepare for getting the compressed_data
|
409
|
+
# Prepare for getting the compressed_data
|
306
410
|
stream[:next_out] = output_buffer_p
|
307
411
|
stream[:avail_out] = CHUNK_SIZE
|
308
412
|
|
309
|
-
#Compress the data
|
413
|
+
# Compress the data
|
310
414
|
res = if io.eof?
|
311
415
|
LibLZMA.lzma_code(stream.pointer, LibLZMA::LZMA_ACTION[:lzma_finish])
|
312
416
|
else
|
@@ -314,26 +418,27 @@ module XZ
|
|
314
418
|
end
|
315
419
|
check_lzma_code_retval(res)
|
316
420
|
|
317
|
-
#Write the compressed data
|
421
|
+
# Write the compressed data
|
318
422
|
data = output_buffer_p.read_string(CHUNK_SIZE - stream[:avail_out])
|
319
423
|
yield(data)
|
320
424
|
|
321
|
-
#If the buffer is completely filled, it's likely that there
|
322
|
-
#more data liblzma wants to hand to us. Start a new
|
323
|
-
#but don't provide new input data.
|
425
|
+
# If the buffer is completely filled, it's likely that there
|
426
|
+
# is more data liblzma wants to hand to us. Start a new
|
427
|
+
# iteration, but don't provide new input data.
|
324
428
|
break unless stream[:avail_out] == 0
|
325
429
|
end #loop
|
326
430
|
end #while
|
327
431
|
end #lzma_code
|
328
432
|
|
329
|
-
#Checks for errors and warnings that can be derived from the
|
330
|
-
#value of the lzma_code() function and shows them if
|
433
|
+
# Checks for errors and warnings that can be derived from the
|
434
|
+
# return value of the lzma_code() function and shows them if
|
435
|
+
# necessary.
|
331
436
|
def check_lzma_code_retval(code)
|
332
437
|
e = LibLZMA::LZMA_RET
|
333
438
|
case code
|
334
|
-
when e[:lzma_no_check] then warn("Couldn't verify archive integrity--archive has
|
439
|
+
when e[:lzma_no_check] then warn("Couldn't verify archive integrity--archive has no integrity checksum.")
|
335
440
|
when e[:lzma_unsupported_check] then warn("Couldn't verify archive integrity--archive has an unsupported integrity checksum.")
|
336
|
-
when e[:lzma_get_check] then nil #This isn't useful for us. It indicates that the checksum type is now known.
|
441
|
+
when e[:lzma_get_check] then nil # This isn't useful for us. It indicates that the checksum type is now known.
|
337
442
|
else
|
338
443
|
LZMAError.raise_if_necessary(code)
|
339
444
|
end
|
@@ -343,6 +448,7 @@ module XZ
|
|
343
448
|
|
344
449
|
end
|
345
450
|
|
451
|
+
require_relative "xz/version"
|
346
452
|
require_relative "xz/lib_lzma"
|
347
453
|
require_relative "xz/stream"
|
348
454
|
require_relative "xz/stream_writer"
|