extzstd 0.0.3.CONCEPT → 0.3.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/HISTORY.ja.md +39 -0
- data/LICENSE +6 -6
- data/README.md +26 -45
- data/contrib/zstd/CHANGELOG +555 -0
- data/contrib/zstd/CODE_OF_CONDUCT.md +5 -0
- data/contrib/zstd/CONTRIBUTING.md +392 -0
- data/contrib/zstd/COPYING +339 -0
- data/contrib/zstd/LICENSE +13 -9
- data/contrib/zstd/Makefile +414 -0
- data/contrib/zstd/README.md +170 -45
- data/contrib/zstd/TESTING.md +44 -0
- data/contrib/zstd/appveyor.yml +289 -0
- data/contrib/zstd/lib/BUCK +234 -0
- data/contrib/zstd/lib/Makefile +354 -0
- data/contrib/zstd/lib/README.md +179 -0
- data/contrib/zstd/{common → lib/common}/bitstream.h +170 -130
- data/contrib/zstd/lib/common/compiler.h +175 -0
- data/contrib/zstd/lib/common/cpu.h +215 -0
- data/contrib/zstd/lib/common/debug.c +24 -0
- data/contrib/zstd/lib/common/debug.h +114 -0
- data/contrib/zstd/{common → lib/common}/entropy_common.c +79 -94
- data/contrib/zstd/lib/common/error_private.c +55 -0
- data/contrib/zstd/lib/common/error_private.h +80 -0
- data/contrib/zstd/{common → lib/common}/fse.h +153 -93
- data/contrib/zstd/{common → lib/common}/fse_decompress.c +37 -82
- data/contrib/zstd/lib/common/huf.h +340 -0
- data/contrib/zstd/{common → lib/common}/mem.h +154 -78
- data/contrib/zstd/lib/common/pool.c +344 -0
- data/contrib/zstd/lib/common/pool.h +84 -0
- data/contrib/zstd/lib/common/threading.c +121 -0
- data/contrib/zstd/lib/common/threading.h +155 -0
- data/contrib/zstd/{common → lib/common}/xxhash.c +85 -75
- data/contrib/zstd/{common → lib/common}/xxhash.h +85 -73
- data/contrib/zstd/lib/common/zstd_common.c +83 -0
- data/contrib/zstd/lib/common/zstd_errors.h +94 -0
- data/contrib/zstd/lib/common/zstd_internal.h +447 -0
- data/contrib/zstd/{compress → lib/compress}/fse_compress.c +194 -303
- data/contrib/zstd/lib/compress/hist.c +183 -0
- data/contrib/zstd/lib/compress/hist.h +75 -0
- data/contrib/zstd/lib/compress/huf_compress.c +798 -0
- data/contrib/zstd/lib/compress/zstd_compress.c +4278 -0
- data/contrib/zstd/lib/compress/zstd_compress_internal.h +1125 -0
- data/contrib/zstd/lib/compress/zstd_compress_literals.c +158 -0
- data/contrib/zstd/lib/compress/zstd_compress_literals.h +29 -0
- data/contrib/zstd/lib/compress/zstd_compress_sequences.c +419 -0
- data/contrib/zstd/lib/compress/zstd_compress_sequences.h +54 -0
- data/contrib/zstd/lib/compress/zstd_compress_superblock.c +845 -0
- data/contrib/zstd/lib/compress/zstd_compress_superblock.h +32 -0
- data/contrib/zstd/lib/compress/zstd_cwksp.h +525 -0
- data/contrib/zstd/lib/compress/zstd_double_fast.c +521 -0
- data/contrib/zstd/lib/compress/zstd_double_fast.h +38 -0
- data/contrib/zstd/lib/compress/zstd_fast.c +496 -0
- data/contrib/zstd/lib/compress/zstd_fast.h +37 -0
- data/contrib/zstd/lib/compress/zstd_lazy.c +1138 -0
- data/contrib/zstd/lib/compress/zstd_lazy.h +67 -0
- data/contrib/zstd/lib/compress/zstd_ldm.c +619 -0
- data/contrib/zstd/lib/compress/zstd_ldm.h +110 -0
- data/contrib/zstd/lib/compress/zstd_opt.c +1200 -0
- data/contrib/zstd/lib/compress/zstd_opt.h +56 -0
- data/contrib/zstd/lib/compress/zstdmt_compress.c +2143 -0
- data/contrib/zstd/lib/compress/zstdmt_compress.h +192 -0
- data/contrib/zstd/lib/decompress/huf_decompress.c +1248 -0
- data/contrib/zstd/lib/decompress/zstd_ddict.c +244 -0
- data/contrib/zstd/lib/decompress/zstd_ddict.h +44 -0
- data/contrib/zstd/lib/decompress/zstd_decompress.c +1885 -0
- data/contrib/zstd/lib/decompress/zstd_decompress_block.c +1432 -0
- data/contrib/zstd/lib/decompress/zstd_decompress_block.h +59 -0
- data/contrib/zstd/lib/decompress/zstd_decompress_internal.h +189 -0
- data/contrib/zstd/{common → lib/deprecated}/zbuff.h +86 -69
- data/contrib/zstd/lib/deprecated/zbuff_common.c +26 -0
- data/contrib/zstd/lib/deprecated/zbuff_compress.c +147 -0
- data/contrib/zstd/lib/deprecated/zbuff_decompress.c +75 -0
- data/contrib/zstd/lib/dictBuilder/cover.c +1236 -0
- data/contrib/zstd/lib/dictBuilder/cover.h +157 -0
- data/contrib/zstd/{dictBuilder → lib/dictBuilder}/divsufsort.c +3 -3
- data/contrib/zstd/{dictBuilder → lib/dictBuilder}/divsufsort.h +5 -5
- data/contrib/zstd/lib/dictBuilder/fastcover.c +757 -0
- data/contrib/zstd/{dictBuilder → lib/dictBuilder}/zdict.c +437 -347
- data/contrib/zstd/lib/dictBuilder/zdict.h +305 -0
- data/contrib/zstd/lib/legacy/zstd_legacy.h +415 -0
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v01.c +272 -292
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v01.h +26 -32
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v02.c +162 -392
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v02.h +26 -32
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v03.c +162 -391
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v03.h +27 -33
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v04.c +195 -604
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v04.h +26 -32
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v05.c +300 -575
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v05.h +22 -31
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v06.c +165 -592
- data/contrib/zstd/{legacy → lib/legacy}/zstd_v06.h +54 -67
- data/contrib/zstd/lib/legacy/zstd_v07.c +4541 -0
- data/contrib/zstd/lib/legacy/zstd_v07.h +187 -0
- data/contrib/zstd/lib/libzstd.pc.in +15 -0
- data/contrib/zstd/lib/zstd.h +2090 -0
- data/ext/depend +2 -0
- data/ext/extconf.rb +18 -5
- data/ext/extzstd.c +296 -214
- data/ext/extzstd.h +81 -36
- data/ext/extzstd_nogvls.h +0 -117
- data/ext/extzstd_stream.c +622 -0
- data/ext/libzstd_conf.h +8 -0
- data/ext/zstd_common.c +11 -0
- data/ext/zstd_compress.c +15 -0
- data/ext/zstd_decompress.c +6 -0
- data/ext/zstd_dictbuilder.c +10 -0
- data/ext/zstd_dictbuilder_fastcover.c +3 -0
- data/ext/zstd_legacy_v01.c +3 -1
- data/ext/zstd_legacy_v02.c +3 -1
- data/ext/zstd_legacy_v03.c +3 -1
- data/ext/zstd_legacy_v04.c +3 -1
- data/ext/zstd_legacy_v05.c +3 -1
- data/ext/zstd_legacy_v06.c +3 -1
- data/ext/zstd_legacy_v07.c +3 -0
- data/gemstub.rb +27 -21
- data/lib/extzstd.rb +82 -161
- data/lib/extzstd/version.rb +1 -1
- data/test/test_basic.rb +19 -6
- metadata +127 -59
- data/contrib/zstd/common/error_private.h +0 -125
- data/contrib/zstd/common/error_public.h +0 -77
- data/contrib/zstd/common/huf.h +0 -228
- data/contrib/zstd/common/zstd.h +0 -475
- data/contrib/zstd/common/zstd_common.c +0 -91
- data/contrib/zstd/common/zstd_internal.h +0 -238
- data/contrib/zstd/compress/huf_compress.c +0 -577
- data/contrib/zstd/compress/zbuff_compress.c +0 -327
- data/contrib/zstd/compress/zstd_compress.c +0 -3074
- data/contrib/zstd/compress/zstd_opt.h +0 -1046
- data/contrib/zstd/decompress/huf_decompress.c +0 -894
- data/contrib/zstd/decompress/zbuff_decompress.c +0 -294
- data/contrib/zstd/decompress/zstd_decompress.c +0 -1362
- data/contrib/zstd/dictBuilder/zdict.h +0 -113
- data/contrib/zstd/legacy/zstd_legacy.h +0 -140
- data/ext/extzstd_buffered.c +0 -265
- data/ext/zstd_amalgam.c +0 -18
@@ -0,0 +1,354 @@
|
|
1
|
+
# ################################################################
|
2
|
+
# Copyright (c) 2015-2020, Yann Collet, Facebook, Inc.
|
3
|
+
# All rights reserved.
|
4
|
+
#
|
5
|
+
# This source code is licensed under both the BSD-style license (found in the
|
6
|
+
# LICENSE file in the root directory of this source tree) and the GPLv2 (found
|
7
|
+
# in the COPYING file in the root directory of this source tree).
|
8
|
+
# You may select, at your option, one of the above-listed licenses.
|
9
|
+
# ################################################################
|
10
|
+
|
11
|
+
Q = $(if $(filter 1,$(V) $(VERBOSE)),,@)
|
12
|
+
|
13
|
+
# When cross-compiling from linux to windows, you might
|
14
|
+
# need to specify this as "Windows." Fedora build fails
|
15
|
+
# without it.
|
16
|
+
#
|
17
|
+
# Note: mingw-w64 build from linux to windows does not
|
18
|
+
# fail on other tested distros (ubuntu, debian) even
|
19
|
+
# without manually specifying the TARGET_SYSTEM.
|
20
|
+
TARGET_SYSTEM ?= $(OS)
|
21
|
+
|
22
|
+
# Version numbers
|
23
|
+
LIBVER_MAJOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MAJOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./zstd.h`
|
24
|
+
LIBVER_MINOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MINOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./zstd.h`
|
25
|
+
LIBVER_PATCH_SCRIPT:=`sed -n '/define ZSTD_VERSION_RELEASE/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./zstd.h`
|
26
|
+
LIBVER_SCRIPT:= $(LIBVER_MAJOR_SCRIPT).$(LIBVER_MINOR_SCRIPT).$(LIBVER_PATCH_SCRIPT)
|
27
|
+
LIBVER_MAJOR := $(shell echo $(LIBVER_MAJOR_SCRIPT))
|
28
|
+
LIBVER_MINOR := $(shell echo $(LIBVER_MINOR_SCRIPT))
|
29
|
+
LIBVER_PATCH := $(shell echo $(LIBVER_PATCH_SCRIPT))
|
30
|
+
LIBVER := $(shell echo $(LIBVER_SCRIPT))
|
31
|
+
VERSION?= $(LIBVER)
|
32
|
+
CCVER := $(shell $(CC) --version)
|
33
|
+
|
34
|
+
CPPFLAGS+= -DXXH_NAMESPACE=ZSTD_
|
35
|
+
ifeq ($(TARGET_SYSTEM),Windows_NT) # MinGW assumed
|
36
|
+
CPPFLAGS += -D__USE_MINGW_ANSI_STDIO # compatibility with %zu formatting
|
37
|
+
endif
|
38
|
+
DEBUGFLAGS= -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow \
|
39
|
+
-Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement \
|
40
|
+
-Wstrict-prototypes -Wundef -Wpointer-arith \
|
41
|
+
-Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings \
|
42
|
+
-Wredundant-decls -Wmissing-prototypes -Wc++-compat
|
43
|
+
CFLAGS += $(DEBUGFLAGS) $(MOREFLAGS)
|
44
|
+
FLAGS = $(CPPFLAGS) $(CFLAGS)
|
45
|
+
|
46
|
+
HAVE_COLORNEVER = $(shell echo a | grep --color=never a > /dev/null 2> /dev/null && echo 1 || echo 0)
|
47
|
+
GREP_OPTIONS ?=
|
48
|
+
ifeq ($HAVE_COLORNEVER, 1)
|
49
|
+
GREP_OPTIONS += --color=never
|
50
|
+
endif
|
51
|
+
GREP = grep $(GREP_OPTIONS)
|
52
|
+
|
53
|
+
ZSTDCOMMON_FILES := $(sort $(wildcard common/*.c))
|
54
|
+
ZSTDCOMP_FILES := $(sort $(wildcard compress/*.c))
|
55
|
+
ZSTDDECOMP_FILES := $(sort $(wildcard decompress/*.c))
|
56
|
+
ZDICT_FILES := $(sort $(wildcard dictBuilder/*.c))
|
57
|
+
ZDEPR_FILES := $(sort $(wildcard deprecated/*.c))
|
58
|
+
ZSTD_FILES := $(ZSTDCOMMON_FILES)
|
59
|
+
|
60
|
+
ifeq ($(findstring GCC,$(CCVER)),GCC)
|
61
|
+
decompress/zstd_decompress_block.o : CFLAGS+=-fno-tree-vectorize
|
62
|
+
endif
|
63
|
+
|
64
|
+
# This is a helper variable that configures a bunch of other variables to new,
|
65
|
+
# space-optimized defaults.
|
66
|
+
ZSTD_LIB_MINIFY ?= 0
|
67
|
+
ifneq ($(ZSTD_LIB_MINIFY), 0)
|
68
|
+
HAVE_CC_OZ ?= $(shell echo "" | $(CC) -Oz -x c -c - -o /dev/null 2> /dev/null && echo 1 || echo 0)
|
69
|
+
ZSTD_LEGACY_SUPPORT ?= 0
|
70
|
+
ZSTD_LIB_DEPRECATED ?= 0
|
71
|
+
HUF_FORCE_DECOMPRESS_X1 ?= 1
|
72
|
+
ZSTD_FORCE_DECOMPRESS_SHORT ?= 1
|
73
|
+
ZSTD_NO_INLINE ?= 1
|
74
|
+
ZSTD_STRIP_ERROR_STRINGS ?= 1
|
75
|
+
ifneq ($(HAVE_CC_OZ), 0)
|
76
|
+
# Some compilers (clang) support an even more space-optimized setting.
|
77
|
+
CFLAGS += -Oz
|
78
|
+
else
|
79
|
+
CFLAGS += -Os
|
80
|
+
endif
|
81
|
+
CFLAGS += -fno-stack-protector -fomit-frame-pointer -fno-ident \
|
82
|
+
-DDYNAMIC_BMI2=0 -DNDEBUG
|
83
|
+
else
|
84
|
+
CFLAGS += -O3
|
85
|
+
endif
|
86
|
+
|
87
|
+
# Modules
|
88
|
+
ZSTD_LIB_COMPRESSION ?= 1
|
89
|
+
ZSTD_LIB_DECOMPRESSION ?= 1
|
90
|
+
ZSTD_LIB_DICTBUILDER ?= 1
|
91
|
+
ZSTD_LIB_DEPRECATED ?= 1
|
92
|
+
|
93
|
+
# Legacy support
|
94
|
+
ZSTD_LEGACY_SUPPORT ?= 5
|
95
|
+
ZSTD_LEGACY_MULTITHREADED_API ?= 0
|
96
|
+
|
97
|
+
# Build size optimizations
|
98
|
+
HUF_FORCE_DECOMPRESS_X1 ?= 0
|
99
|
+
HUF_FORCE_DECOMPRESS_X2 ?= 0
|
100
|
+
ZSTD_FORCE_DECOMPRESS_SHORT ?= 0
|
101
|
+
ZSTD_FORCE_DECOMPRESS_LONG ?= 0
|
102
|
+
ZSTD_NO_INLINE ?= 0
|
103
|
+
ZSTD_STRIP_ERROR_STRINGS ?= 0
|
104
|
+
|
105
|
+
ifeq ($(ZSTD_LIB_COMPRESSION), 0)
|
106
|
+
ZSTD_LIB_DICTBUILDER = 0
|
107
|
+
ZSTD_LIB_DEPRECATED = 0
|
108
|
+
endif
|
109
|
+
|
110
|
+
ifeq ($(ZSTD_LIB_DECOMPRESSION), 0)
|
111
|
+
ZSTD_LEGACY_SUPPORT = 0
|
112
|
+
ZSTD_LIB_DEPRECATED = 0
|
113
|
+
endif
|
114
|
+
|
115
|
+
ifneq ($(ZSTD_LIB_COMPRESSION), 0)
|
116
|
+
ZSTD_FILES += $(ZSTDCOMP_FILES)
|
117
|
+
endif
|
118
|
+
|
119
|
+
ifneq ($(ZSTD_LIB_DECOMPRESSION), 0)
|
120
|
+
ZSTD_FILES += $(ZSTDDECOMP_FILES)
|
121
|
+
endif
|
122
|
+
|
123
|
+
ifneq ($(ZSTD_LIB_DEPRECATED), 0)
|
124
|
+
ZSTD_FILES += $(ZDEPR_FILES)
|
125
|
+
endif
|
126
|
+
|
127
|
+
ifneq ($(ZSTD_LIB_DICTBUILDER), 0)
|
128
|
+
ZSTD_FILES += $(ZDICT_FILES)
|
129
|
+
endif
|
130
|
+
|
131
|
+
ifneq ($(HUF_FORCE_DECOMPRESS_X1), 0)
|
132
|
+
CFLAGS += -DHUF_FORCE_DECOMPRESS_X1
|
133
|
+
endif
|
134
|
+
|
135
|
+
ifneq ($(HUF_FORCE_DECOMPRESS_X2), 0)
|
136
|
+
CFLAGS += -DHUF_FORCE_DECOMPRESS_X2
|
137
|
+
endif
|
138
|
+
|
139
|
+
ifneq ($(ZSTD_FORCE_DECOMPRESS_SHORT), 0)
|
140
|
+
CFLAGS += -DZSTD_FORCE_DECOMPRESS_SHORT
|
141
|
+
endif
|
142
|
+
|
143
|
+
ifneq ($(ZSTD_FORCE_DECOMPRESS_LONG), 0)
|
144
|
+
CFLAGS += -DZSTD_FORCE_DECOMPRESS_LONG
|
145
|
+
endif
|
146
|
+
|
147
|
+
ifneq ($(ZSTD_NO_INLINE), 0)
|
148
|
+
CFLAGS += -DZSTD_NO_INLINE
|
149
|
+
endif
|
150
|
+
|
151
|
+
ifneq ($(ZSTD_STRIP_ERROR_STRINGS), 0)
|
152
|
+
CFLAGS += -DZSTD_STRIP_ERROR_STRINGS
|
153
|
+
endif
|
154
|
+
|
155
|
+
ifneq ($(ZSTD_LEGACY_MULTITHREADED_API), 0)
|
156
|
+
CFLAGS += -DZSTD_LEGACY_MULTITHREADED_API
|
157
|
+
endif
|
158
|
+
|
159
|
+
ifneq ($(ZSTD_LEGACY_SUPPORT), 0)
|
160
|
+
ifeq ($(shell test $(ZSTD_LEGACY_SUPPORT) -lt 8; echo $$?), 0)
|
161
|
+
ZSTD_FILES += $(shell ls legacy/*.c | $(GREP) 'v0[$(ZSTD_LEGACY_SUPPORT)-7]')
|
162
|
+
endif
|
163
|
+
endif
|
164
|
+
CPPFLAGS += -DZSTD_LEGACY_SUPPORT=$(ZSTD_LEGACY_SUPPORT)
|
165
|
+
|
166
|
+
ZSTD_OBJ := $(patsubst %.c,%.o,$(ZSTD_FILES))
|
167
|
+
|
168
|
+
# macOS linker doesn't support -soname, and use different extension
|
169
|
+
# see : https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html
|
170
|
+
ifeq ($(shell uname), Darwin)
|
171
|
+
SHARED_EXT = dylib
|
172
|
+
SHARED_EXT_MAJOR = $(LIBVER_MAJOR).$(SHARED_EXT)
|
173
|
+
SHARED_EXT_VER = $(LIBVER).$(SHARED_EXT)
|
174
|
+
SONAME_FLAGS = -install_name $(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR) -compatibility_version $(LIBVER_MAJOR) -current_version $(LIBVER)
|
175
|
+
else
|
176
|
+
SONAME_FLAGS = -Wl,-soname=libzstd.$(SHARED_EXT).$(LIBVER_MAJOR)
|
177
|
+
SHARED_EXT = so
|
178
|
+
SHARED_EXT_MAJOR = $(SHARED_EXT).$(LIBVER_MAJOR)
|
179
|
+
SHARED_EXT_VER = $(SHARED_EXT).$(LIBVER)
|
180
|
+
endif
|
181
|
+
|
182
|
+
|
183
|
+
.PHONY: default lib-all all clean install uninstall
|
184
|
+
|
185
|
+
default: lib-release
|
186
|
+
|
187
|
+
# alias
|
188
|
+
lib-all: all
|
189
|
+
|
190
|
+
all: lib
|
191
|
+
|
192
|
+
libzstd.a: ARFLAGS = rcs
|
193
|
+
libzstd.a: $(ZSTD_OBJ)
|
194
|
+
@echo compiling static library
|
195
|
+
$(Q)$(AR) $(ARFLAGS) $@ $^
|
196
|
+
|
197
|
+
ifneq (,$(filter Windows%,$(TARGET_SYSTEM)))
|
198
|
+
|
199
|
+
LIBZSTD = dll\libzstd.dll
|
200
|
+
$(LIBZSTD): $(ZSTD_FILES)
|
201
|
+
@echo compiling dynamic library $(LIBVER)
|
202
|
+
$(CC) $(FLAGS) -DZSTD_DLL_EXPORT=1 -Wl,--out-implib,dll\libzstd.dll.a -shared $^ -o $@
|
203
|
+
|
204
|
+
else
|
205
|
+
|
206
|
+
LIBZSTD = libzstd.$(SHARED_EXT_VER)
|
207
|
+
$(LIBZSTD): LDFLAGS += -shared -fPIC -fvisibility=hidden
|
208
|
+
$(LIBZSTD): $(ZSTD_FILES)
|
209
|
+
@echo compiling dynamic library $(LIBVER)
|
210
|
+
$(Q)$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@
|
211
|
+
@echo creating versioned links
|
212
|
+
$(Q)ln -sf $@ libzstd.$(SHARED_EXT_MAJOR)
|
213
|
+
$(Q)ln -sf $@ libzstd.$(SHARED_EXT)
|
214
|
+
|
215
|
+
endif
|
216
|
+
|
217
|
+
.PHONY: libzstd
|
218
|
+
libzstd : $(LIBZSTD)
|
219
|
+
|
220
|
+
.PHONY: lib
|
221
|
+
lib : libzstd.a libzstd
|
222
|
+
|
223
|
+
.PHONY: lib-mt
|
224
|
+
%-mt : CPPFLAGS += -DZSTD_MULTITHREAD
|
225
|
+
%-mt : LDFLAGS += -pthread
|
226
|
+
%-mt : %
|
227
|
+
@echo multi-threading build completed
|
228
|
+
|
229
|
+
.PHONY: lib-release
|
230
|
+
%-release : DEBUGFLAGS :=
|
231
|
+
%-release : %
|
232
|
+
@echo release build completed
|
233
|
+
|
234
|
+
|
235
|
+
# Special case : building library in single-thread mode _and_ without zstdmt_compress.c
|
236
|
+
ZSTDMT_FILES = compress/zstdmt_compress.c
|
237
|
+
ZSTD_NOMT_FILES = $(filter-out $(ZSTDMT_FILES),$(ZSTD_FILES))
|
238
|
+
libzstd-nomt: LDFLAGS += -shared -fPIC -fvisibility=hidden
|
239
|
+
libzstd-nomt: $(ZSTD_NOMT_FILES)
|
240
|
+
@echo compiling single-thread dynamic library $(LIBVER)
|
241
|
+
@echo files : $(ZSTD_NOMT_FILES)
|
242
|
+
$(Q)$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@
|
243
|
+
|
244
|
+
clean:
|
245
|
+
$(Q)$(RM) -r *.dSYM # macOS-specific
|
246
|
+
$(Q)$(RM) core *.o *.a *.gcda *.$(SHARED_EXT) *.$(SHARED_EXT).* libzstd.pc
|
247
|
+
$(Q)$(RM) dll/libzstd.dll dll/libzstd.lib libzstd-nomt*
|
248
|
+
$(Q)$(RM) common/*.o compress/*.o decompress/*.o dictBuilder/*.o legacy/*.o deprecated/*.o
|
249
|
+
@echo Cleaning library completed
|
250
|
+
|
251
|
+
#-----------------------------------------------------------------------------
|
252
|
+
# make install is validated only for below listed environments
|
253
|
+
#-----------------------------------------------------------------------------
|
254
|
+
ifneq (,$(filter $(shell uname),Linux Darwin GNU/kFreeBSD GNU OpenBSD FreeBSD NetBSD DragonFly SunOS Haiku))
|
255
|
+
|
256
|
+
all: libzstd.pc
|
257
|
+
|
258
|
+
DESTDIR ?=
|
259
|
+
# directory variables : GNU conventions prefer lowercase
|
260
|
+
# see https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html
|
261
|
+
# support both lower and uppercase (BSD), use uppercase in script
|
262
|
+
prefix ?= /usr/local
|
263
|
+
PREFIX ?= $(prefix)
|
264
|
+
exec_prefix ?= $(PREFIX)
|
265
|
+
EXEC_PREFIX ?= $(exec_prefix)
|
266
|
+
libdir ?= $(EXEC_PREFIX)/lib
|
267
|
+
LIBDIR ?= $(libdir)
|
268
|
+
includedir ?= $(PREFIX)/include
|
269
|
+
INCLUDEDIR ?= $(includedir)
|
270
|
+
|
271
|
+
PCLIBDIR ?= $(shell echo "$(LIBDIR)" | sed -n -E -e "s@^$(EXEC_PREFIX)(/|$$)@@p")
|
272
|
+
PCINCDIR ?= $(shell echo "$(INCLUDEDIR)" | sed -n -E -e "s@^$(PREFIX)(/|$$)@@p")
|
273
|
+
|
274
|
+
ifeq (,$(PCLIBDIR))
|
275
|
+
# Additional prefix check is required, since the empty string is technically a
|
276
|
+
# valid PCLIBDIR
|
277
|
+
ifeq (,$(shell echo "$(LIBDIR)" | sed -n -E -e "\\@^$(EXEC_PREFIX)(/|$$)@ p"))
|
278
|
+
$(error configured libdir ($(LIBDIR)) is outside of prefix ($(PREFIX)), can't generate pkg-config file)
|
279
|
+
endif
|
280
|
+
endif
|
281
|
+
|
282
|
+
ifeq (,$(PCINCDIR))
|
283
|
+
# Additional prefix check is required, since the empty string is technically a
|
284
|
+
# valid PCINCDIR
|
285
|
+
ifeq (,$(shell echo "$(INCLUDEDIR)" | sed -n -E -e "\\@^$(PREFIX)(/|$$)@ p"))
|
286
|
+
$(error configured includedir ($(INCLUDEDIR)) is outside of exec_prefix ($(EXEC_PREFIX)), can't generate pkg-config file)
|
287
|
+
endif
|
288
|
+
endif
|
289
|
+
|
290
|
+
ifneq (,$(filter $(shell uname),FreeBSD NetBSD DragonFly))
|
291
|
+
PKGCONFIGDIR ?= $(PREFIX)/libdata/pkgconfig
|
292
|
+
else
|
293
|
+
PKGCONFIGDIR ?= $(LIBDIR)/pkgconfig
|
294
|
+
endif
|
295
|
+
|
296
|
+
ifneq (,$(filter $(shell uname),SunOS))
|
297
|
+
INSTALL ?= ginstall
|
298
|
+
else
|
299
|
+
INSTALL ?= install
|
300
|
+
endif
|
301
|
+
|
302
|
+
INSTALL_PROGRAM ?= $(INSTALL)
|
303
|
+
INSTALL_DATA ?= $(INSTALL) -m 644
|
304
|
+
|
305
|
+
|
306
|
+
libzstd.pc:
|
307
|
+
libzstd.pc: libzstd.pc.in
|
308
|
+
@echo creating pkgconfig
|
309
|
+
$(Q)@sed -E -e 's|@PREFIX@|$(PREFIX)|' \
|
310
|
+
-e 's|@LIBDIR@|$(PCLIBDIR)|' \
|
311
|
+
-e 's|@INCLUDEDIR@|$(PCINCDIR)|' \
|
312
|
+
-e 's|@VERSION@|$(VERSION)|' \
|
313
|
+
$< >$@
|
314
|
+
|
315
|
+
install: install-pc install-static install-shared install-includes
|
316
|
+
@echo zstd static and shared library installed
|
317
|
+
|
318
|
+
install-pc: libzstd.pc
|
319
|
+
$(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(PKGCONFIGDIR)/
|
320
|
+
$(Q)$(INSTALL_DATA) libzstd.pc $(DESTDIR)$(PKGCONFIGDIR)/
|
321
|
+
|
322
|
+
install-static: libzstd.a
|
323
|
+
@echo Installing static library
|
324
|
+
$(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/
|
325
|
+
$(Q)$(INSTALL_DATA) libzstd.a $(DESTDIR)$(LIBDIR)
|
326
|
+
|
327
|
+
install-shared: libzstd
|
328
|
+
@echo Installing shared library
|
329
|
+
$(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/
|
330
|
+
$(Q)$(INSTALL_PROGRAM) $(LIBZSTD) $(DESTDIR)$(LIBDIR)
|
331
|
+
$(Q)ln -sf $(LIBZSTD) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR)
|
332
|
+
$(Q)ln -sf $(LIBZSTD) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT)
|
333
|
+
|
334
|
+
install-includes:
|
335
|
+
@echo Installing includes
|
336
|
+
$(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(INCLUDEDIR)/
|
337
|
+
$(Q)$(INSTALL_DATA) zstd.h $(DESTDIR)$(INCLUDEDIR)
|
338
|
+
$(Q)$(INSTALL_DATA) common/zstd_errors.h $(DESTDIR)$(INCLUDEDIR)
|
339
|
+
$(Q)$(INSTALL_DATA) deprecated/zbuff.h $(DESTDIR)$(INCLUDEDIR) # prototypes generate deprecation warnings
|
340
|
+
$(Q)$(INSTALL_DATA) dictBuilder/zdict.h $(DESTDIR)$(INCLUDEDIR)
|
341
|
+
|
342
|
+
uninstall:
|
343
|
+
$(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.a
|
344
|
+
$(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT)
|
345
|
+
$(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR)
|
346
|
+
$(Q)$(RM) $(DESTDIR)$(LIBDIR)/$(LIBZSTD)
|
347
|
+
$(Q)$(RM) $(DESTDIR)$(PKGCONFIGDIR)/libzstd.pc
|
348
|
+
$(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zstd.h
|
349
|
+
$(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zstd_errors.h
|
350
|
+
$(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zbuff.h # Deprecated streaming functions
|
351
|
+
$(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zdict.h
|
352
|
+
@echo zstd libraries successfully uninstalled
|
353
|
+
|
354
|
+
endif
|
@@ -0,0 +1,179 @@
|
|
1
|
+
Zstandard library files
|
2
|
+
================================
|
3
|
+
|
4
|
+
The __lib__ directory is split into several sub-directories,
|
5
|
+
in order to make it easier to select or exclude features.
|
6
|
+
|
7
|
+
|
8
|
+
#### Building
|
9
|
+
|
10
|
+
`Makefile` script is provided, supporting [Makefile conventions](https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html#Makefile-Conventions),
|
11
|
+
including commands variables, staged install, directory variables and standard targets.
|
12
|
+
- `make` : generates both static and dynamic libraries
|
13
|
+
- `make install` : install libraries and headers in target system directories
|
14
|
+
|
15
|
+
`libzstd` default scope is pretty large, including compression, decompression, dictionary builder,
|
16
|
+
and support for decoding legacy formats >= v0.5.0.
|
17
|
+
The scope can be reduced on demand (see paragraph _modular build_).
|
18
|
+
|
19
|
+
|
20
|
+
#### Multithreading support
|
21
|
+
|
22
|
+
Multithreading is disabled by default when building with `make`.
|
23
|
+
Enabling multithreading requires 2 conditions :
|
24
|
+
- set build macro `ZSTD_MULTITHREAD` (`-DZSTD_MULTITHREAD` for `gcc`)
|
25
|
+
- for POSIX systems : compile with pthread (`-pthread` compilation flag for `gcc`)
|
26
|
+
|
27
|
+
Both conditions are automatically applied when invoking `make lib-mt` target.
|
28
|
+
|
29
|
+
When linking a POSIX program with a multithreaded version of `libzstd`,
|
30
|
+
note that it's necessary to invoke the `-pthread` flag during link stage.
|
31
|
+
|
32
|
+
Multithreading capabilities are exposed
|
33
|
+
via the [advanced API defined in `lib/zstd.h`](https://github.com/facebook/zstd/blob/v1.4.3/lib/zstd.h#L351).
|
34
|
+
|
35
|
+
|
36
|
+
#### API
|
37
|
+
|
38
|
+
Zstandard's stable API is exposed within [lib/zstd.h](zstd.h).
|
39
|
+
|
40
|
+
|
41
|
+
#### Advanced API
|
42
|
+
|
43
|
+
Optional advanced features are exposed via :
|
44
|
+
|
45
|
+
- `lib/common/zstd_errors.h` : translates `size_t` function results
|
46
|
+
into a `ZSTD_ErrorCode`, for accurate error handling.
|
47
|
+
|
48
|
+
- `ZSTD_STATIC_LINKING_ONLY` : if this macro is defined _before_ including `zstd.h`,
|
49
|
+
it unlocks access to the experimental API,
|
50
|
+
exposed in the second part of `zstd.h`.
|
51
|
+
All definitions in the experimental APIs are unstable,
|
52
|
+
they may still change in the future, or even be removed.
|
53
|
+
As a consequence, experimental definitions shall ___never be used with dynamic library___ !
|
54
|
+
Only static linking is allowed.
|
55
|
+
|
56
|
+
|
57
|
+
#### Modular build
|
58
|
+
|
59
|
+
It's possible to compile only a limited set of features within `libzstd`.
|
60
|
+
The file structure is designed to make this selection manually achievable for any build system :
|
61
|
+
|
62
|
+
- Directory `lib/common` is always required, for all variants.
|
63
|
+
|
64
|
+
- Compression source code lies in `lib/compress`
|
65
|
+
|
66
|
+
- Decompression source code lies in `lib/decompress`
|
67
|
+
|
68
|
+
- It's possible to include only `compress` or only `decompress`, they don't depend on each other.
|
69
|
+
|
70
|
+
- `lib/dictBuilder` : makes it possible to generate dictionaries from a set of samples.
|
71
|
+
The API is exposed in `lib/dictBuilder/zdict.h`.
|
72
|
+
This module depends on both `lib/common` and `lib/compress` .
|
73
|
+
|
74
|
+
- `lib/legacy` : makes it possible to decompress legacy zstd formats, starting from `v0.1.0`.
|
75
|
+
This module depends on `lib/common` and `lib/decompress`.
|
76
|
+
To enable this feature, define `ZSTD_LEGACY_SUPPORT` during compilation.
|
77
|
+
Specifying a number limits versions supported to that version onward.
|
78
|
+
For example, `ZSTD_LEGACY_SUPPORT=2` means : "support legacy formats >= v0.2.0".
|
79
|
+
Conversely, `ZSTD_LEGACY_SUPPORT=0` means "do __not__ support legacy formats".
|
80
|
+
By default, this build macro is set as `ZSTD_LEGACY_SUPPORT=5`.
|
81
|
+
Decoding supported legacy format is a transparent capability triggered within decompression functions.
|
82
|
+
It's also allowed to invoke legacy API directly, exposed in `lib/legacy/zstd_legacy.h`.
|
83
|
+
Each version does also provide its own set of advanced API.
|
84
|
+
For example, advanced API for version `v0.4` is exposed in `lib/legacy/zstd_v04.h` .
|
85
|
+
|
86
|
+
- While invoking `make libzstd`, it's possible to define build macros
|
87
|
+
`ZSTD_LIB_COMPRESSION, ZSTD_LIB_DECOMPRESSION`, `ZSTD_LIB_DICTBUILDER`,
|
88
|
+
and `ZSTD_LIB_DEPRECATED` as `0` to forgo compilation of the
|
89
|
+
corresponding features. This will also disable compilation of all
|
90
|
+
dependencies (eg. `ZSTD_LIB_COMPRESSION=0` will also disable
|
91
|
+
dictBuilder).
|
92
|
+
|
93
|
+
- There are a number of options that can help minimize the binary size of
|
94
|
+
`libzstd`.
|
95
|
+
|
96
|
+
The first step is to select the components needed (using the above-described
|
97
|
+
`ZSTD_LIB_COMPRESSION` etc.).
|
98
|
+
|
99
|
+
The next step is to set `ZSTD_LIB_MINIFY` to `1` when invoking `make`. This
|
100
|
+
disables various optional components and changes the compilation flags to
|
101
|
+
prioritize space-saving.
|
102
|
+
|
103
|
+
Detailed options: Zstandard's code and build environment is set up by default
|
104
|
+
to optimize above all else for performance. In pursuit of this goal, Zstandard
|
105
|
+
makes significant trade-offs in code size. For example, Zstandard often has
|
106
|
+
more than one implementation of a particular component, with each
|
107
|
+
implementation optimized for different scenarios. For example, the Huffman
|
108
|
+
decoder has complementary implementations that decode the stream one symbol at
|
109
|
+
a time or two symbols at a time. Zstd normally includes both (and dispatches
|
110
|
+
between them at runtime), but by defining `HUF_FORCE_DECOMPRESS_X1` or
|
111
|
+
`HUF_FORCE_DECOMPRESS_X2`, you can force the use of one or the other, avoiding
|
112
|
+
compilation of the other. Similarly, `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT`
|
113
|
+
and `ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG` force the compilation and use of
|
114
|
+
only one or the other of two decompression implementations. The smallest
|
115
|
+
binary is achieved by using `HUF_FORCE_DECOMPRESS_X1` and
|
116
|
+
`ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT` (implied by `ZSTD_LIB_MINIFY`).
|
117
|
+
|
118
|
+
For squeezing the last ounce of size out, you can also define
|
119
|
+
`ZSTD_NO_INLINE`, which disables inlining, and `ZSTD_STRIP_ERROR_STRINGS`,
|
120
|
+
which removes the error messages that are otherwise returned by
|
121
|
+
`ZSTD_getErrorName` (implied by `ZSTD_LIB_MINIFY`).
|
122
|
+
|
123
|
+
Finally, when integrating into your application, make sure you're doing link-
|
124
|
+
time optimation and unused symbol garbage collection (via some combination of,
|
125
|
+
e.g., `-flto`, `-ffat-lto-objects`, `-fuse-linker-plugin`,
|
126
|
+
`-ffunction-sections`, `-fdata-sections`, `-fmerge-all-constants`,
|
127
|
+
`-Wl,--gc-sections`, `-Wl,-z,norelro`, and an archiver that understands
|
128
|
+
the compiler's intermediate representation, e.g., `AR=gcc-ar`). Consult your
|
129
|
+
compiler's documentation.
|
130
|
+
|
131
|
+
- While invoking `make libzstd`, the build macro `ZSTD_LEGACY_MULTITHREADED_API=1`
|
132
|
+
will expose the deprecated `ZSTDMT` API exposed by `zstdmt_compress.h` in
|
133
|
+
the shared library, which is now hidden by default.
|
134
|
+
|
135
|
+
- The build macro `DYNAMIC_BMI2` can be set to 1 or 0 in order to generate binaries
|
136
|
+
which can detect at runtime the presence of BMI2 instructions, and use them only if present.
|
137
|
+
These instructions contribute to better performance, notably on the decoder side.
|
138
|
+
By default, this feature is automatically enabled on detecting
|
139
|
+
the right instruction set (x64) and compiler (clang or gcc >= 5).
|
140
|
+
It's obviously disabled for different cpus,
|
141
|
+
or when BMI2 instruction set is _required_ by the compiler command line
|
142
|
+
(in this case, only the BMI2 code path is generated).
|
143
|
+
Setting this macro will either force to generate the BMI2 dispatcher (1)
|
144
|
+
or prevent it (0). It overrides automatic detection.
|
145
|
+
|
146
|
+
|
147
|
+
#### Windows : using MinGW+MSYS to create DLL
|
148
|
+
|
149
|
+
DLL can be created using MinGW+MSYS with the `make libzstd` command.
|
150
|
+
This command creates `dll\libzstd.dll` and the import library `dll\libzstd.lib`.
|
151
|
+
The import library is only required with Visual C++.
|
152
|
+
The header file `zstd.h` and the dynamic library `dll\libzstd.dll` are required to
|
153
|
+
compile a project using gcc/MinGW.
|
154
|
+
The dynamic library has to be added to linking options.
|
155
|
+
It means that if a project that uses ZSTD consists of a single `test-dll.c`
|
156
|
+
file it should be linked with `dll\libzstd.dll`. For example:
|
157
|
+
```
|
158
|
+
gcc $(CFLAGS) -Iinclude/ test-dll.c -o test-dll dll\libzstd.dll
|
159
|
+
```
|
160
|
+
The compiled executable will require ZSTD DLL which is available at `dll\libzstd.dll`.
|
161
|
+
|
162
|
+
|
163
|
+
#### Deprecated API
|
164
|
+
|
165
|
+
Obsolete API on their way out are stored in directory `lib/deprecated`.
|
166
|
+
At this stage, it contains older streaming prototypes, in `lib/deprecated/zbuff.h`.
|
167
|
+
These prototypes will be removed in some future version.
|
168
|
+
Consider migrating code towards supported streaming API exposed in `zstd.h`.
|
169
|
+
|
170
|
+
|
171
|
+
#### Miscellaneous
|
172
|
+
|
173
|
+
The other files are not source code. There are :
|
174
|
+
|
175
|
+
- `BUCK` : support for `buck` build system (https://buckbuild.com/)
|
176
|
+
- `Makefile` : `make` script to build and install zstd library (static and dynamic)
|
177
|
+
- `README.md` : this file
|
178
|
+
- `dll/` : resources directory for Windows compilation
|
179
|
+
- `libzstd.pc.in` : script for `pkg-config` (used in `make install`)
|