extlz4 0.2.4.2 → 0.3.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/HISTORY.ja.md +25 -0
- data/README.md +49 -41
- data/bin/extlz4 +1 -1
- data/contrib/lz4/INSTALL +1 -0
- data/contrib/lz4/Makefile.inc +87 -0
- data/contrib/lz4/NEWS +89 -0
- data/contrib/lz4/README.md +42 -36
- data/contrib/lz4/build/README.md +55 -0
- data/contrib/lz4/build/VS2010/datagen/datagen.vcxproj +169 -0
- data/contrib/lz4/build/VS2010/frametest/frametest.vcxproj +176 -0
- data/contrib/lz4/build/VS2010/fullbench-dll/fullbench-dll.vcxproj +180 -0
- data/contrib/lz4/build/VS2010/fullbench/fullbench.vcxproj +176 -0
- data/contrib/lz4/build/VS2010/fuzzer/fuzzer.vcxproj +173 -0
- data/contrib/lz4/build/VS2010/liblz4-dll/liblz4-dll.rc +51 -0
- data/contrib/lz4/build/VS2010/liblz4-dll/liblz4-dll.vcxproj +179 -0
- data/contrib/lz4/build/VS2010/liblz4/liblz4.vcxproj +175 -0
- data/contrib/lz4/build/VS2010/lz4.sln +98 -0
- data/contrib/lz4/build/VS2010/lz4/lz4.rc +51 -0
- data/contrib/lz4/build/VS2010/lz4/lz4.vcxproj +189 -0
- data/contrib/lz4/build/VS2017/datagen/datagen.vcxproj +173 -0
- data/contrib/lz4/build/VS2017/frametest/frametest.vcxproj +180 -0
- data/contrib/lz4/build/VS2017/fullbench-dll/fullbench-dll.vcxproj +184 -0
- data/contrib/lz4/build/VS2017/fullbench/fullbench.vcxproj +180 -0
- data/contrib/lz4/build/VS2017/fuzzer/fuzzer.vcxproj +177 -0
- data/contrib/lz4/build/VS2017/liblz4-dll/liblz4-dll.rc +51 -0
- data/contrib/lz4/build/VS2017/liblz4-dll/liblz4-dll.vcxproj +183 -0
- data/contrib/lz4/build/VS2017/liblz4/liblz4.vcxproj +179 -0
- data/contrib/lz4/build/VS2017/lz4.sln +103 -0
- data/contrib/lz4/build/VS2017/lz4/lz4.rc +51 -0
- data/contrib/lz4/build/VS2017/lz4/lz4.vcxproj +164 -0
- data/contrib/lz4/build/cmake/CMakeLists.txt +235 -0
- data/contrib/lz4/lib/README.md +98 -34
- data/contrib/lz4/lib/liblz4-dll.rc.in +35 -0
- data/contrib/lz4/lib/lz4.c +1698 -681
- data/contrib/lz4/lib/lz4.h +546 -235
- data/contrib/lz4/lib/lz4frame.c +608 -378
- data/contrib/lz4/lib/lz4frame.h +315 -83
- data/contrib/lz4/lib/lz4frame_static.h +4 -100
- data/contrib/lz4/lib/lz4hc.c +1090 -282
- data/contrib/lz4/lib/lz4hc.h +276 -141
- data/contrib/lz4/lib/xxhash.c +371 -235
- data/contrib/lz4/lib/xxhash.h +128 -93
- data/contrib/lz4/ossfuzz/Makefile +78 -0
- data/contrib/lz4/ossfuzz/compress_frame_fuzzer.c +48 -0
- data/contrib/lz4/ossfuzz/compress_fuzzer.c +58 -0
- data/contrib/lz4/ossfuzz/compress_hc_fuzzer.c +64 -0
- data/contrib/lz4/ossfuzz/decompress_frame_fuzzer.c +75 -0
- data/contrib/lz4/ossfuzz/decompress_fuzzer.c +62 -0
- data/contrib/lz4/ossfuzz/fuzz.h +48 -0
- data/contrib/lz4/ossfuzz/fuzz_data_producer.c +77 -0
- data/contrib/lz4/ossfuzz/fuzz_data_producer.h +36 -0
- data/contrib/lz4/ossfuzz/fuzz_helpers.h +94 -0
- data/contrib/lz4/ossfuzz/lz4_helpers.c +51 -0
- data/contrib/lz4/ossfuzz/lz4_helpers.h +13 -0
- data/contrib/lz4/ossfuzz/ossfuzz.sh +23 -0
- data/contrib/lz4/ossfuzz/round_trip_frame_fuzzer.c +43 -0
- data/contrib/lz4/ossfuzz/round_trip_fuzzer.c +57 -0
- data/contrib/lz4/ossfuzz/round_trip_hc_fuzzer.c +44 -0
- data/contrib/lz4/ossfuzz/round_trip_stream_fuzzer.c +302 -0
- data/contrib/lz4/ossfuzz/standaloneengine.c +74 -0
- data/contrib/lz4/ossfuzz/travisoss.sh +26 -0
- data/contrib/lz4/tmp +0 -0
- data/contrib/lz4/tmpsparse +0 -0
- data/ext/blockapi.c +5 -5
- data/ext/extlz4.c +2 -0
- data/ext/extlz4.h +5 -0
- data/ext/frameapi.c +1 -1
- data/ext/hashargs.c +2 -2
- data/ext/hashargs.h +1 -1
- data/ext/lz4_amalgam.c +0 -23
- data/gemstub.rb +5 -16
- data/lib/extlz4.rb +51 -3
- data/lib/extlz4/oldstream.rb +1 -1
- data/test/common.rb +2 -2
- metadata +73 -16
- data/contrib/lz4/circle.yml +0 -39
- data/contrib/lz4/lib/lz4opt.h +0 -366
- data/lib/extlz4/version.rb +0 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: f1060d0000526f937ddf36e247c7e2041fbdf7241382d38a88a85bd39738d958
|
4
|
+
data.tar.gz: 61e5321caa13e409b8bd1af20429c184fca8fcc87f73ce31b1cbd4caf1acde24
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 487f4884327c7111ad64877190fb07a25a38ff1de84b1544ec131a238d6b1f617daf89d6ff5a9776ec05783d78139c41c481f8b2fd7b2dfba56c92019fbce413
|
7
|
+
data.tar.gz: 00c9a7ccdec52f3a7f60aeae9626af3f83ef8a7a8a456fe267bb4244e930fe977d2c9af72137e2e6fb1cc6906b5ab7a84eb01a393c03e69cc83d393284eae710
|
data/HISTORY.ja.md
CHANGED
@@ -1,3 +1,28 @@
|
|
1
|
+
# extlz4-0.3.1 (令和2年6月7日)
|
2
|
+
|
3
|
+
* lz4-1.9.2 に更新
|
4
|
+
* `extlz4.so` が `<extlz4-dir>/lib` へ置かれない環境に対応
|
5
|
+
* https://github.com/dearblue/ruby-extlz4/issues/2
|
6
|
+
|
7
|
+
|
8
|
+
# extlz4-0.3 (平成31年4月)
|
9
|
+
|
10
|
+
* lz4-1.9.0 に更新
|
11
|
+
* `LZ4` モジュールをリファインメントとして利用できるように機能を追加
|
12
|
+
* `Object` および `String` クラスで `.to_lz4frame` `.unlz4frame` メソッドが利用できるようになりました。
|
13
|
+
* `String` クラスで `.to_lz4block` `.unlz4block` メソッドが利用できるようになりました。
|
14
|
+
|
15
|
+
|
16
|
+
# extlz4-0.2.5 (平成30年1月14日 日曜日)
|
17
|
+
|
18
|
+
* lz4 ライブラリを [1.8.1](https://github.com/lz4/lz4/releases/tag/v1.8.1) に更新
|
19
|
+
|
20
|
+
|
21
|
+
# extlz4-0.2.4.3 (平成30年1月13日 土曜日)
|
22
|
+
|
23
|
+
* LZ4::Decoder.read でバッファ領域が確保されていなかった問題を修正
|
24
|
+
|
25
|
+
|
1
26
|
# extlz4-0.2.4.2 (平成30年1月7日 日曜日)
|
2
27
|
|
3
28
|
* バージョン番号の改定のみ
|
data/README.md
CHANGED
@@ -5,28 +5,28 @@
|
|
5
5
|
|
6
6
|
LZ4 データストリームを圧縮・伸張できます。lz4-cli で扱うことが出来ます。
|
7
7
|
|
8
|
-
```
|
8
|
+
```shell:shell
|
9
9
|
$ dmesg | ruby -r extlz4 -e 'LZ4.encode_file($stdin.binmode, $stdout.binmode)' | lz4c -d | more
|
10
10
|
```
|
11
11
|
|
12
12
|
ほかの ruby 向けの lz4 バインディングライブラリとしては KOMIYA Atsushi さんによる [lz4-ruby (http://rubygems.org/gems/lz4-ruby)](http://rubygems.org/gems/lz4-ruby) があります。
|
13
13
|
|
14
14
|
|
15
|
-
##
|
16
|
-
|
17
|
-
*
|
18
|
-
|
19
|
-
|
20
|
-
*
|
21
|
-
|
22
|
-
|
23
|
-
*
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
*
|
28
|
-
|
29
|
-
|
15
|
+
## FEATURES (機能)
|
16
|
+
|
17
|
+
* Generic LZ4 frame data process (for `.lz4` file format)
|
18
|
+
* Decode LZ4 Frame data : `LZ4.decode`
|
19
|
+
* Encode LZ4 Frame data : `LZ4.encode`
|
20
|
+
* Generic LZ4 frame data file process (for `.lz4` file format)
|
21
|
+
* Decode LZ4 Frame data file : `LZ4.decode_file`
|
22
|
+
* Encode LZ4 Frame data file : `LZ4.encode_file`
|
23
|
+
* LZ4 block data process
|
24
|
+
* Decode LZ4 block data : `LZ4.block_decode`
|
25
|
+
* Encode LZ4 block data : `LZ4.block_encode` (supporting high compression level)
|
26
|
+
* Streaming Decode LZ4 block data : `LZ4.block_stream_decode` and `LZ4::BlockDecoder#update`
|
27
|
+
* Streaming Encode LZ4 block data : `LZ4.block_stream_encode` and `LZ4::BlockEncoder#update` (supporting high compression level)
|
28
|
+
|
29
|
+
See [Quick reference](QUICKREF.md) for more details.
|
30
30
|
|
31
31
|
|
32
32
|
## ATTENTIONS (注意事項)
|
@@ -59,21 +59,6 @@ extlz4-0.1.1 でその不具合の修正を行いました。
|
|
59
59
|
修復できるのはあくまで extlz4-0.1 のこの不具合に起因するファイルのみとなります。
|
60
60
|
|
61
61
|
|
62
|
-
## FEATURES (機能)
|
63
|
-
|
64
|
-
* Generic LZ4 frame data process
|
65
|
-
* Decode LZ4 Frame data : LZ4.decode
|
66
|
-
* Encode LZ4 Frame data : LZ4.encode
|
67
|
-
* Generic LZ4 frame data file process
|
68
|
-
* Decode LZ4 Frame data file : LZ4.decode\_file
|
69
|
-
* Encode LZ4 Frame data file : LZ4.encode\_file
|
70
|
-
* LZ4 block data process
|
71
|
-
* Decode LZ4 block data : LZ4.block\_decode
|
72
|
-
* Encode LZ4 block data : LZ4.block\_encode (supporting high compression level)
|
73
|
-
* Streaming Decode LZ4 block data : LZ4.block\_stream\_decode and LZ4::BlockDecoder#update
|
74
|
-
* Streaming Encode LZ4 block data : LZ4.block\_stream\_encode and LZ4::BlockEncoder#update (supporting high compression level)
|
75
|
-
|
76
|
-
|
77
62
|
## ABOUT TAINT STATE AND SECURITY (汚染状態とセキュリティについて)
|
78
63
|
|
79
64
|
extlz4 はオブジェクト間での汚染状態を一方向伝播します。
|
@@ -92,25 +77,25 @@ First, load extlz4. (最初に extlz4 を読み込んでください)
|
|
92
77
|
require "extlz4"
|
93
78
|
```
|
94
79
|
|
95
|
-
###
|
80
|
+
### One shot decoding from LZ4 Frame (LZ4 Frame 伸張処理)
|
96
81
|
|
97
82
|
``` ruby:ruby
|
98
83
|
uncompressed_data_string = LZ4.decode(compressed_data_string)
|
99
84
|
```
|
100
85
|
|
101
|
-
###
|
86
|
+
### One shot encoding to LZ4 Frame (LZ4 Frame 通常圧縮処理)
|
102
87
|
|
103
88
|
``` ruby:ruby
|
104
89
|
compressed_data_string = LZ4.encode(uncompressed_data_string)
|
105
90
|
```
|
106
91
|
|
107
|
-
###
|
92
|
+
### One shot high compression encoding to LZ4 Frame (LZ4 Frame 高圧縮処理)
|
108
93
|
|
109
94
|
``` ruby:ruby
|
110
95
|
compressed_data_string = LZ4.encode(uncompressed_data_string, 9)
|
111
96
|
```
|
112
97
|
|
113
|
-
###
|
98
|
+
### Stream decoding to LZ4 Frame
|
114
99
|
|
115
100
|
``` ruby:ruby
|
116
101
|
File.open("sample.txt.lz4", "rb") do |file|
|
@@ -122,7 +107,7 @@ File.open("sample.txt.lz4", "rb") do |file|
|
|
122
107
|
end
|
123
108
|
```
|
124
109
|
|
125
|
-
###
|
110
|
+
### Stream encoding by high compression to LZ4 Frame
|
126
111
|
|
127
112
|
``` ruby:ruby
|
128
113
|
File.open("sample.txt.lz4", "wb") do |file|
|
@@ -133,7 +118,7 @@ File.open("sample.txt.lz4", "wb") do |file|
|
|
133
118
|
end
|
134
119
|
```
|
135
120
|
|
136
|
-
###
|
121
|
+
### Stream encoding without block to LZ4 Frame
|
137
122
|
|
138
123
|
``` ruby:ruby
|
139
124
|
file = File.open("sample.txt.lz4", "wb")
|
@@ -142,7 +127,7 @@ lz4 << "abcdefghijklmnopqrstuvwxyz\n"
|
|
142
127
|
lz4.close # VERY IMPORTANT!
|
143
128
|
```
|
144
129
|
|
145
|
-
###
|
130
|
+
### One shot block data processing to/from LZ4 Block (fast compression encoding and decoding)
|
146
131
|
|
147
132
|
``` ruby:ruby
|
148
133
|
src = "abcdefg" * 100
|
@@ -151,7 +136,7 @@ data = LZ4.block_decode(lz4data)
|
|
151
136
|
p src == data # => true
|
152
137
|
```
|
153
138
|
|
154
|
-
###
|
139
|
+
### One shot block data processing to/from LZ4 Block (high compression encoding and decoding)
|
155
140
|
|
156
141
|
``` ruby:ruby
|
157
142
|
src = "abcdefg" * 100
|
@@ -161,7 +146,7 @@ data = LZ4.block_decode(lz4data)
|
|
161
146
|
p src == data # => true
|
162
147
|
```
|
163
148
|
|
164
|
-
###
|
149
|
+
### One shot block data processing to/from LZ4 Block (high speed encoding)
|
165
150
|
|
166
151
|
``` ruby:ruby
|
167
152
|
src = "abcdefg" * 100
|
@@ -169,7 +154,7 @@ level = -19 # transform to one's complement as acceleration
|
|
169
154
|
lz4data = LZ4.block_encode(level, src)
|
170
155
|
```
|
171
156
|
|
172
|
-
### Block stream data processing (high compression encoding and decoding)
|
157
|
+
### Block stream data processing to/from LZ4 Block (high compression encoding and decoding)
|
173
158
|
|
174
159
|
``` ruby:ruby
|
175
160
|
level = 8 # (OPTIONAL PARAMETER)
|
@@ -194,6 +179,11 @@ p src2 == data # => true
|
|
194
179
|
```
|
195
180
|
|
196
181
|
|
182
|
+
## Support `Ractor` with Ruby3
|
183
|
+
|
184
|
+
Ruby3 で追加された `Ractor` に対応しています。
|
185
|
+
|
186
|
+
|
197
187
|
## BONUS (おまけ)
|
198
188
|
|
199
189
|
コマンドラインプログラムとして ``extlz4`` が追加されます。
|
@@ -201,3 +191,21 @@ p src2 == data # => true
|
|
201
191
|
これは lz4 と同程度の機能を持ちます (車輪の再発明とも言う)。
|
202
192
|
|
203
193
|
とはいえ、引数のとり方を変えてあり、gzip のような形で利用できます。
|
194
|
+
|
195
|
+
|
196
|
+
## SPECIFICATION (仕様)
|
197
|
+
|
198
|
+
- package name: extlz4
|
199
|
+
- author: dearblue (mailto:dearblue@users.osdn.me)
|
200
|
+
- project page: <https://github.com/dearblue/ruby-extlz4>
|
201
|
+
- how to install: `gem install extlz4`
|
202
|
+
- version: 0.3.2
|
203
|
+
- product quality: technical preview
|
204
|
+
- licensing: [2 clause BSD License](LICENSE)
|
205
|
+
- dependency gems: none
|
206
|
+
- dependency external c libraries: none
|
207
|
+
- bundled external c libraries (git submodules):
|
208
|
+
- [lz4](https://github.com/lz4/lz4)
|
209
|
+
[version 1.9.3](https://github.com/lz4/lz4/tree/v1.9.3)
|
210
|
+
under [2 clause BSD license](https://github.com/lz4/lz4/blob/v1.9.3/LICENSE)
|
211
|
+
by [Yann Collet](https://github.com/Cyan4973)
|
data/bin/extlz4
CHANGED
data/contrib/lz4/INSTALL
CHANGED
@@ -8,6 +8,7 @@ make install # this command may require root access
|
|
8
8
|
|
9
9
|
LZ4's `Makefile` supports standard [Makefile conventions],
|
10
10
|
including [staged installs], [redirection], or [command redefinition].
|
11
|
+
It is compatible with parallel builds (`-j#`).
|
11
12
|
|
12
13
|
[Makefile conventions]: https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html
|
13
14
|
[staged installs]: https://www.gnu.org/prep/standards/html_node/DESTDIR.html
|
@@ -0,0 +1,87 @@
|
|
1
|
+
ifeq ($(V), 1)
|
2
|
+
Q =
|
3
|
+
else
|
4
|
+
Q = @
|
5
|
+
endif
|
6
|
+
|
7
|
+
TARGET_OS ?= $(shell uname)
|
8
|
+
ifeq ($(TARGET_OS),)
|
9
|
+
TARGET_OS ?= $(OS)
|
10
|
+
endif
|
11
|
+
|
12
|
+
ifneq (,$(filter Windows%,$(TARGET_OS)))
|
13
|
+
LIBLZ4 = liblz4-$(LIBVER_MAJOR)
|
14
|
+
LIBLZ4_EXP = liblz4.lib
|
15
|
+
WINBASED = yes
|
16
|
+
else
|
17
|
+
LIBLZ4_EXP = liblz4.dll.a
|
18
|
+
ifneq (,$(filter MINGW%,$(TARGET_OS)))
|
19
|
+
LIBLZ4 = liblz4
|
20
|
+
WINBASED = yes
|
21
|
+
else
|
22
|
+
ifneq (,$(filter MSYS%,$(TARGET_OS)))
|
23
|
+
LIBLZ4 = msys-lz4-$(LIBVER_MAJOR)
|
24
|
+
WINBASED = yes
|
25
|
+
else
|
26
|
+
ifneq (,$(filter CYGWIN%,$(TARGET_OS)))
|
27
|
+
LIBLZ4 = cyglz4-$(LIBVER_MAJOR)
|
28
|
+
WINBASED = yes
|
29
|
+
else
|
30
|
+
LIBLZ4 = liblz4.$(SHARED_EXT_VER)
|
31
|
+
WINBASED = no
|
32
|
+
EXT =
|
33
|
+
endif
|
34
|
+
endif
|
35
|
+
endif
|
36
|
+
endif
|
37
|
+
|
38
|
+
ifeq ($(WINBASED),yes)
|
39
|
+
EXT = .exe
|
40
|
+
WINDRES = windres
|
41
|
+
endif
|
42
|
+
|
43
|
+
#determine if dev/nul based on host environment
|
44
|
+
ifneq (,$(filter MINGW% MSYS% CYGWIN%,$(shell uname)))
|
45
|
+
VOID := /dev/null
|
46
|
+
else
|
47
|
+
ifneq (,$(filter Windows%,$(OS)))
|
48
|
+
VOID := nul
|
49
|
+
else
|
50
|
+
VOID := /dev/null
|
51
|
+
endif
|
52
|
+
endif
|
53
|
+
|
54
|
+
ifneq (,$(filter Linux Darwin GNU/kFreeBSD GNU OpenBSD FreeBSD NetBSD DragonFly SunOS Haiku MidnightBSD MINGW% CYGWIN% MSYS%,$(shell uname)))
|
55
|
+
POSIX_ENV = Yes
|
56
|
+
else
|
57
|
+
POSIX_ENV = No
|
58
|
+
endif
|
59
|
+
|
60
|
+
# Avoid symlinks when targetting Windows or building on a Windows host
|
61
|
+
ifeq ($(WINBASED),yes)
|
62
|
+
LN_S = cp -p
|
63
|
+
LN_SF = cp -p
|
64
|
+
else
|
65
|
+
ifneq (,$(filter MINGW% MSYS% CYGWIN%,$(shell uname)))
|
66
|
+
LN_S = cp -p
|
67
|
+
LN_SF = cp -p
|
68
|
+
else
|
69
|
+
ifneq (,$(filter Windows%,$(OS)))
|
70
|
+
LN_S = cp -p
|
71
|
+
LN_SF = cp -p
|
72
|
+
else
|
73
|
+
LN_S = ln -s
|
74
|
+
LN_SF = ln -sf
|
75
|
+
endif
|
76
|
+
endif
|
77
|
+
endif
|
78
|
+
|
79
|
+
ifneq (,$(filter $(shell uname),SunOS))
|
80
|
+
INSTALL ?= ginstall
|
81
|
+
else
|
82
|
+
INSTALL ?= install
|
83
|
+
endif
|
84
|
+
|
85
|
+
INSTALL_PROGRAM ?= $(INSTALL) -m 755
|
86
|
+
INSTALL_DATA ?= $(INSTALL) -m 644
|
87
|
+
INSTALL_DIR ?= $(INSTALL) -d -m 755
|
data/contrib/lz4/NEWS
CHANGED
@@ -1,3 +1,92 @@
|
|
1
|
+
v1.9.3
|
2
|
+
perf: highly improved speed in kernel space, by @terrelln
|
3
|
+
perf: faster speed with Visual Studio, thanks to @wolfpld and @remittor
|
4
|
+
perf: improved dictionary compression speed, by @felixhandte
|
5
|
+
perf: fixed LZ4_compress_HC_destSize() ratio, detected by @hsiangkao
|
6
|
+
perf: reduced stack usage in high compression mode, by @Yanpas
|
7
|
+
api : LZ4_decompress_safe_partial() supports unknown compressed size, requested by @jfkthame
|
8
|
+
api : improved LZ4F_compressBound() with automatic flushing, by Christopher Harvie
|
9
|
+
api : can (de)compress to/from NULL without UBs
|
10
|
+
api : fix alignment test on 32-bit systems (state initialization)
|
11
|
+
api : fix LZ4_saveDictHC() in corner case scenario, detected by @IgorKorkin
|
12
|
+
cli : `-l` legacy format is now compatible with `-m` multiple files, by Filipe Calasans
|
13
|
+
cli : benchmark mode supports dictionary, by @rkoradi
|
14
|
+
cli : fix --fast with large argument, detected by @picoHz
|
15
|
+
build: link to user-defined memory functions with LZ4_USER_MEMORY_FUNCTIONS, suggested by Yuriy Levchenko
|
16
|
+
build: contrib/cmake_unofficial/ moved to build/cmake/
|
17
|
+
build: visual/* moved to build/
|
18
|
+
build: updated meson script, by @neheb
|
19
|
+
build: tinycc support, by Anton Kochkov
|
20
|
+
install: Haiku support, by Jerome Duval
|
21
|
+
doc : updated LZ4 frame format, clarify EndMark
|
22
|
+
|
23
|
+
v1.9.2
|
24
|
+
fix : out-of-bound read in exceptional circumstances when using decompress_partial(), by @terrelln
|
25
|
+
fix : slim opportunity for out-of-bound write with compress_fast() with a large enough input and when providing an output smaller than recommended (< LZ4_compressBound(inputSize)), by @terrelln
|
26
|
+
fix : rare data corruption bug with LZ4_compress_destSize(), by @terrelln
|
27
|
+
fix : data corruption bug when Streaming with an Attached Dict in HC Mode, by @felixhandte
|
28
|
+
perf: enable LZ4_FAST_DEC_LOOP on aarch64/GCC by default, by @prekageo
|
29
|
+
perf: improved lz4frame streaming API speed, by @dreambottle
|
30
|
+
perf: speed up lz4hc on slow patterns when using external dictionary, by @terrelln
|
31
|
+
api: better in-place decompression and compression support
|
32
|
+
cli : --list supports multi-frames files, by @gstedman
|
33
|
+
cli: --version outputs to stdout
|
34
|
+
cli : add option --best as an alias of -12 , by @Low-power
|
35
|
+
misc: Integration into oss-fuzz by @cmeister2, expanded list of scenarios by @terrelln
|
36
|
+
|
37
|
+
v1.9.1
|
38
|
+
fix : decompression functions were reading a few bytes beyond input size (introduced in v1.9.0, reported by @ppodolsky and @danlark1)
|
39
|
+
api : fix : lz4frame initializers compatibility with c++, reported by @degski
|
40
|
+
cli : added command --list, based on a patch by @gabrielstedman
|
41
|
+
build: improved Windows build, by @JPeterMugaas
|
42
|
+
build: AIX, by Norman Green
|
43
|
+
|
44
|
+
v1.9.0
|
45
|
+
perf: large decompression speed improvement on x86/x64 (up to +20%) by @djwatson
|
46
|
+
api : changed : _destSize() compression variants are promoted to stable API
|
47
|
+
api : new : LZ4_initStream(HC), replacing LZ4_resetStream(HC)
|
48
|
+
api : changed : LZ4_resetStream(HC) as recommended reset function, for better performance on small data
|
49
|
+
cli : support custom block sizes, by @blezsan
|
50
|
+
build: source code can be amalgamated, by Bing Xu
|
51
|
+
build: added meson build, by @lzutao
|
52
|
+
build: new build macros : LZ4_DISTANCE_MAX, LZ4_FAST_DEC_LOOP
|
53
|
+
install: MidnightBSD, by @laffer1
|
54
|
+
install: msys2 on Windows 10, by @vtorri
|
55
|
+
|
56
|
+
v1.8.3
|
57
|
+
perf: minor decompression speed improvement (~+2%) with gcc
|
58
|
+
fix : corruption in v1.8.2 at level 9 for files > 64KB under rare conditions (#560)
|
59
|
+
cli : new command --fast, by @jennifermliu
|
60
|
+
cli : fixed elapsed time, and added cpu load indicator (on -vv) (#555)
|
61
|
+
api : LZ4_decompress_safe_partial() now decodes exactly the nb of bytes requested (feature request #566)
|
62
|
+
build : added Haiku target, by @fbrosson, and MidnightBSD, by @laffer1
|
63
|
+
doc : updated documentation regarding dictionary compression
|
64
|
+
|
65
|
+
v1.8.2
|
66
|
+
perf: *much* faster dictionary compression on small files, by @felixhandte
|
67
|
+
perf: improved decompression speed and binary size, by Alexey Tourbin (@svpv)
|
68
|
+
perf: slightly faster HC compression and decompression speed
|
69
|
+
perf: very small compression ratio improvement
|
70
|
+
fix : compression compatible with low memory addresses (< 0xFFFF)
|
71
|
+
fix : decompression segfault when provided with NULL input, by @terrelln
|
72
|
+
cli : new command --favor-decSpeed
|
73
|
+
cli : benchmark mode more accurate for small inputs
|
74
|
+
fullbench : can bench _destSize() variants, by @felixhandte
|
75
|
+
doc : clarified block format parsing restrictions, by Alexey Tourbin (@svpv)
|
76
|
+
|
77
|
+
v1.8.1
|
78
|
+
perf : faster and stronger ultra modes (levels 10+)
|
79
|
+
perf : slightly faster compression and decompression speed
|
80
|
+
perf : fix bad degenerative case, reported by @c-morgenstern
|
81
|
+
fix : decompression failed when using a combination of extDict + low memory address (#397), reported and fixed by Julian Scheid (@jscheid)
|
82
|
+
cli : support for dictionary compression (`-D`), by Felix Handte @felixhandte
|
83
|
+
cli : fix : `lz4 -d --rm` preserves timestamp (#441)
|
84
|
+
cli : fix : do not modify /dev/null permission as root, by @aliceatlas
|
85
|
+
api : `_destSize()` variant supported for all compression levels
|
86
|
+
build : `make` and `make test` compatible with `-jX`, reported by @mwgamera
|
87
|
+
build : can control LZ4LIB_VISIBILITY macro, by @mikir
|
88
|
+
install: fix man page directory (#387), reported by Stuart Cardall (@itoffshore)
|
89
|
+
|
1
90
|
v1.8.0
|
2
91
|
cli : fix : do not modify /dev/null permissions, reported by @Maokaman1
|
3
92
|
cli : added GNU separator -- specifying that all following arguments are files
|
data/contrib/lz4/README.md
CHANGED
@@ -2,74 +2,68 @@ LZ4 - Extremely fast compression
|
|
2
2
|
================================
|
3
3
|
|
4
4
|
LZ4 is lossless compression algorithm,
|
5
|
-
providing compression speed
|
5
|
+
providing compression speed > 500 MB/s per core,
|
6
6
|
scalable with multi-cores CPU.
|
7
7
|
It features an extremely fast decoder,
|
8
8
|
with speed in multiple GB/s per core,
|
9
9
|
typically reaching RAM speed limits on multi-core systems.
|
10
10
|
|
11
11
|
Speed can be tuned dynamically, selecting an "acceleration" factor
|
12
|
-
which trades compression ratio for
|
12
|
+
which trades compression ratio for faster speed.
|
13
13
|
On the other end, a high compression derivative, LZ4_HC, is also provided,
|
14
14
|
trading CPU time for improved compression ratio.
|
15
15
|
All versions feature the same decompression speed.
|
16
16
|
|
17
|
+
LZ4 is also compatible with [dictionary compression](https://github.com/facebook/zstd#the-case-for-small-data-compression),
|
18
|
+
both at [API](https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4frame.h#L481) and [CLI](https://github.com/lz4/lz4/blob/v1.8.3/programs/lz4.1.md#operation-modifiers) levels.
|
19
|
+
It can ingest any input file as dictionary, though only the final 64KB are used.
|
20
|
+
This capability can be combined with the [Zstandard Dictionary Builder](https://github.com/facebook/zstd/blob/v1.3.5/programs/zstd.1.md#dictionary-builder),
|
21
|
+
in order to drastically improve compression performance on small files.
|
22
|
+
|
23
|
+
|
17
24
|
LZ4 library is provided as open-source software using BSD 2-Clause license.
|
18
25
|
|
19
26
|
|
20
27
|
|Branch |Status |
|
21
28
|
|------------|---------|
|
22
|
-
|master | [![Build Status][travisMasterBadge]][travisLink] [![Build status][AppveyorMasterBadge]][AppveyorLink] [![coverity][coverBadge]][coverlink] |
|
23
29
|
|dev | [![Build Status][travisDevBadge]][travisLink] [![Build status][AppveyorDevBadge]][AppveyorLink] |
|
24
30
|
|
25
|
-
[travisMasterBadge]: https://travis-ci.org/lz4/lz4.svg?branch=master "Continuous Integration test suite"
|
26
31
|
[travisDevBadge]: https://travis-ci.org/lz4/lz4.svg?branch=dev "Continuous Integration test suite"
|
27
32
|
[travisLink]: https://travis-ci.org/lz4/lz4
|
28
|
-
[AppveyorMasterBadge]: https://ci.appveyor.com/api/projects/status/github/lz4/lz4?branch=master&svg=true "Windows test suite"
|
29
33
|
[AppveyorDevBadge]: https://ci.appveyor.com/api/projects/status/github/lz4/lz4?branch=dev&svg=true "Windows test suite"
|
30
34
|
[AppveyorLink]: https://ci.appveyor.com/project/YannCollet/lz4-1lndh
|
31
|
-
[coverBadge]: https://scan.coverity.com/projects/4735/badge.svg "Static code analysis of Master branch"
|
32
|
-
[coverlink]: https://scan.coverity.com/projects/4735
|
33
35
|
|
34
|
-
> **Branch Policy:**
|
35
|
-
> - The "master" branch is considered stable, at all times.
|
36
|
-
> - The "dev" branch is the one where all contributions must be merged
|
37
|
-
before being promoted to master.
|
38
|
-
> + If you plan to propose a patch, please commit into the "dev" branch,
|
39
|
-
or its own feature branch.
|
40
|
-
Direct commit to "master" are not permitted.
|
41
36
|
|
42
37
|
Benchmarks
|
43
38
|
-------------------------
|
44
39
|
|
45
40
|
The benchmark uses [lzbench], from @inikep
|
46
|
-
compiled with GCC
|
47
|
-
The reference system uses a Core i7-
|
41
|
+
compiled with GCC v8.2.0 on Linux 64-bits (Ubuntu 4.18.0-17).
|
42
|
+
The reference system uses a Core i7-9700K CPU @ 4.9GHz (w/ turbo boost).
|
48
43
|
Benchmark evaluates the compression of reference [Silesia Corpus]
|
49
44
|
in single-thread mode.
|
50
45
|
|
51
46
|
[lzbench]: https://github.com/inikep/lzbench
|
52
47
|
[Silesia Corpus]: http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia
|
53
48
|
|
54
|
-
| Compressor
|
55
|
-
| ----------
|
56
|
-
| memcpy
|
57
|
-
|**LZ4
|
58
|
-
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
| LZF v3.6
|
63
|
-
|
|
64
|
-
|
65
|
-
| [zlib] deflate 1.2.
|
66
|
-
|**LZ4 HC -9 (v1.7.3)** |**2.720**| 34 MB/s | **3240 MB/s** |
|
67
|
-
| [zlib] deflate 1.2.8 -6| 3.099 | 33 MB/s | 390 MB/s |
|
49
|
+
| Compressor | Ratio | Compression | Decompression |
|
50
|
+
| ---------- | ----- | ----------- | ------------- |
|
51
|
+
| memcpy | 1.000 | 13700 MB/s | 13700 MB/s |
|
52
|
+
|**LZ4 default (v1.9.0)** |**2.101**| **780 MB/s**| **4970 MB/s** |
|
53
|
+
| LZO 2.09 | 2.108 | 670 MB/s | 860 MB/s |
|
54
|
+
| QuickLZ 1.5.0 | 2.238 | 575 MB/s | 780 MB/s |
|
55
|
+
| Snappy 1.1.4 | 2.091 | 565 MB/s | 1950 MB/s |
|
56
|
+
| [Zstandard] 1.4.0 -1 | 2.883 | 515 MB/s | 1380 MB/s |
|
57
|
+
| LZF v3.6 | 2.073 | 415 MB/s | 910 MB/s |
|
58
|
+
| [zlib] deflate 1.2.11 -1| 2.730 | 100 MB/s | 415 MB/s |
|
59
|
+
|**LZ4 HC -9 (v1.9.0)** |**2.721**| 41 MB/s | **4900 MB/s** |
|
60
|
+
| [zlib] deflate 1.2.11 -6| 3.099 | 36 MB/s | 445 MB/s |
|
68
61
|
|
69
62
|
[zlib]: http://www.zlib.net/
|
70
63
|
[Zstandard]: http://www.zstd.net/
|
71
64
|
|
72
|
-
LZ4 is also compatible and
|
65
|
+
LZ4 is also compatible and optimized for x32 mode,
|
66
|
+
for which it provides additional speed performance.
|
73
67
|
|
74
68
|
|
75
69
|
Installation
|
@@ -77,27 +71,39 @@ Installation
|
|
77
71
|
|
78
72
|
```
|
79
73
|
make
|
80
|
-
make install # this command may require root
|
74
|
+
make install # this command may require root permissions
|
81
75
|
```
|
82
76
|
|
83
77
|
LZ4's `Makefile` supports standard [Makefile conventions],
|
84
78
|
including [staged installs], [redirection], or [command redefinition].
|
79
|
+
It is compatible with parallel builds (`-j#`).
|
85
80
|
|
86
81
|
[Makefile conventions]: https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html
|
87
82
|
[staged installs]: https://www.gnu.org/prep/standards/html_node/DESTDIR.html
|
88
83
|
[redirection]: https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
|
89
84
|
[command redefinition]: https://www.gnu.org/prep/standards/html_node/Utilities-in-Makefiles.html
|
90
85
|
|
86
|
+
Building LZ4 - Using vcpkg
|
87
|
+
|
88
|
+
You can download and install LZ4 using the [vcpkg](https://github.com/Microsoft/vcpkg) dependency manager:
|
89
|
+
|
90
|
+
git clone https://github.com/Microsoft/vcpkg.git
|
91
|
+
cd vcpkg
|
92
|
+
./bootstrap-vcpkg.sh
|
93
|
+
./vcpkg integrate install
|
94
|
+
vcpkg install lz4
|
95
|
+
|
96
|
+
The LZ4 port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please [create an issue or pull request](https://github.com/Microsoft/vcpkg) on the vcpkg repository.
|
91
97
|
|
92
98
|
Documentation
|
93
99
|
-------------------------
|
94
100
|
|
95
101
|
The raw LZ4 block compression format is detailed within [lz4_Block_format].
|
96
102
|
|
97
|
-
|
98
|
-
|
99
|
-
|
100
|
-
Interoperable versions of LZ4 must respect
|
103
|
+
Arbitrarily long files or data streams are compressed using multiple blocks,
|
104
|
+
for streaming requirements. These blocks are organized into a frame,
|
105
|
+
defined into [lz4_Frame_format].
|
106
|
+
Interoperable versions of LZ4 must also respect the frame format.
|
101
107
|
|
102
108
|
[lz4_Block_format]: doc/lz4_Block_format.md
|
103
109
|
[lz4_Frame_format]: doc/lz4_Frame_format.md
|