extzstd 0.0.3.CONCEPT → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (138) hide show
  1. checksums.yaml +5 -5
  2. data/HISTORY.ja.md +39 -0
  3. data/LICENSE +6 -6
  4. data/README.md +26 -45
  5. data/contrib/zstd/CHANGELOG +555 -0
  6. data/contrib/zstd/CODE_OF_CONDUCT.md +5 -0
  7. data/contrib/zstd/CONTRIBUTING.md +392 -0
  8. data/contrib/zstd/COPYING +339 -0
  9. data/contrib/zstd/LICENSE +13 -9
  10. data/contrib/zstd/Makefile +414 -0
  11. data/contrib/zstd/README.md +170 -45
  12. data/contrib/zstd/TESTING.md +44 -0
  13. data/contrib/zstd/appveyor.yml +289 -0
  14. data/contrib/zstd/lib/BUCK +234 -0
  15. data/contrib/zstd/lib/Makefile +354 -0
  16. data/contrib/zstd/lib/README.md +179 -0
  17. data/contrib/zstd/{common → lib/common}/bitstream.h +170 -130
  18. data/contrib/zstd/lib/common/compiler.h +175 -0
  19. data/contrib/zstd/lib/common/cpu.h +215 -0
  20. data/contrib/zstd/lib/common/debug.c +24 -0
  21. data/contrib/zstd/lib/common/debug.h +114 -0
  22. data/contrib/zstd/{common → lib/common}/entropy_common.c +79 -94
  23. data/contrib/zstd/lib/common/error_private.c +55 -0
  24. data/contrib/zstd/lib/common/error_private.h +80 -0
  25. data/contrib/zstd/{common → lib/common}/fse.h +153 -93
  26. data/contrib/zstd/{common → lib/common}/fse_decompress.c +37 -82
  27. data/contrib/zstd/lib/common/huf.h +340 -0
  28. data/contrib/zstd/{common → lib/common}/mem.h +154 -78
  29. data/contrib/zstd/lib/common/pool.c +344 -0
  30. data/contrib/zstd/lib/common/pool.h +84 -0
  31. data/contrib/zstd/lib/common/threading.c +121 -0
  32. data/contrib/zstd/lib/common/threading.h +155 -0
  33. data/contrib/zstd/{common → lib/common}/xxhash.c +85 -75
  34. data/contrib/zstd/{common → lib/common}/xxhash.h +85 -73
  35. data/contrib/zstd/lib/common/zstd_common.c +83 -0
  36. data/contrib/zstd/lib/common/zstd_errors.h +94 -0
  37. data/contrib/zstd/lib/common/zstd_internal.h +447 -0
  38. data/contrib/zstd/{compress → lib/compress}/fse_compress.c +194 -303
  39. data/contrib/zstd/lib/compress/hist.c +183 -0
  40. data/contrib/zstd/lib/compress/hist.h +75 -0
  41. data/contrib/zstd/lib/compress/huf_compress.c +798 -0
  42. data/contrib/zstd/lib/compress/zstd_compress.c +4278 -0
  43. data/contrib/zstd/lib/compress/zstd_compress_internal.h +1125 -0
  44. data/contrib/zstd/lib/compress/zstd_compress_literals.c +158 -0
  45. data/contrib/zstd/lib/compress/zstd_compress_literals.h +29 -0
  46. data/contrib/zstd/lib/compress/zstd_compress_sequences.c +419 -0
  47. data/contrib/zstd/lib/compress/zstd_compress_sequences.h +54 -0
  48. data/contrib/zstd/lib/compress/zstd_compress_superblock.c +845 -0
  49. data/contrib/zstd/lib/compress/zstd_compress_superblock.h +32 -0
  50. data/contrib/zstd/lib/compress/zstd_cwksp.h +525 -0
  51. data/contrib/zstd/lib/compress/zstd_double_fast.c +521 -0
  52. data/contrib/zstd/lib/compress/zstd_double_fast.h +38 -0
  53. data/contrib/zstd/lib/compress/zstd_fast.c +496 -0
  54. data/contrib/zstd/lib/compress/zstd_fast.h +37 -0
  55. data/contrib/zstd/lib/compress/zstd_lazy.c +1138 -0
  56. data/contrib/zstd/lib/compress/zstd_lazy.h +67 -0
  57. data/contrib/zstd/lib/compress/zstd_ldm.c +619 -0
  58. data/contrib/zstd/lib/compress/zstd_ldm.h +110 -0
  59. data/contrib/zstd/lib/compress/zstd_opt.c +1200 -0
  60. data/contrib/zstd/lib/compress/zstd_opt.h +56 -0
  61. data/contrib/zstd/lib/compress/zstdmt_compress.c +2143 -0
  62. data/contrib/zstd/lib/compress/zstdmt_compress.h +192 -0
  63. data/contrib/zstd/lib/decompress/huf_decompress.c +1248 -0
  64. data/contrib/zstd/lib/decompress/zstd_ddict.c +244 -0
  65. data/contrib/zstd/lib/decompress/zstd_ddict.h +44 -0
  66. data/contrib/zstd/lib/decompress/zstd_decompress.c +1885 -0
  67. data/contrib/zstd/lib/decompress/zstd_decompress_block.c +1432 -0
  68. data/contrib/zstd/lib/decompress/zstd_decompress_block.h +59 -0
  69. data/contrib/zstd/lib/decompress/zstd_decompress_internal.h +189 -0
  70. data/contrib/zstd/{common → lib/deprecated}/zbuff.h +86 -69
  71. data/contrib/zstd/lib/deprecated/zbuff_common.c +26 -0
  72. data/contrib/zstd/lib/deprecated/zbuff_compress.c +147 -0
  73. data/contrib/zstd/lib/deprecated/zbuff_decompress.c +75 -0
  74. data/contrib/zstd/lib/dictBuilder/cover.c +1236 -0
  75. data/contrib/zstd/lib/dictBuilder/cover.h +157 -0
  76. data/contrib/zstd/{dictBuilder → lib/dictBuilder}/divsufsort.c +3 -3
  77. data/contrib/zstd/{dictBuilder → lib/dictBuilder}/divsufsort.h +5 -5
  78. data/contrib/zstd/lib/dictBuilder/fastcover.c +757 -0
  79. data/contrib/zstd/{dictBuilder → lib/dictBuilder}/zdict.c +437 -347
  80. data/contrib/zstd/lib/dictBuilder/zdict.h +305 -0
  81. data/contrib/zstd/lib/legacy/zstd_legacy.h +415 -0
  82. data/contrib/zstd/{legacy → lib/legacy}/zstd_v01.c +272 -292
  83. data/contrib/zstd/{legacy → lib/legacy}/zstd_v01.h +26 -32
  84. data/contrib/zstd/{legacy → lib/legacy}/zstd_v02.c +162 -392
  85. data/contrib/zstd/{legacy → lib/legacy}/zstd_v02.h +26 -32
  86. data/contrib/zstd/{legacy → lib/legacy}/zstd_v03.c +162 -391
  87. data/contrib/zstd/{legacy → lib/legacy}/zstd_v03.h +27 -33
  88. data/contrib/zstd/{legacy → lib/legacy}/zstd_v04.c +195 -604
  89. data/contrib/zstd/{legacy → lib/legacy}/zstd_v04.h +26 -32
  90. data/contrib/zstd/{legacy → lib/legacy}/zstd_v05.c +300 -575
  91. data/contrib/zstd/{legacy → lib/legacy}/zstd_v05.h +22 -31
  92. data/contrib/zstd/{legacy → lib/legacy}/zstd_v06.c +165 -592
  93. data/contrib/zstd/{legacy → lib/legacy}/zstd_v06.h +54 -67
  94. data/contrib/zstd/lib/legacy/zstd_v07.c +4541 -0
  95. data/contrib/zstd/lib/legacy/zstd_v07.h +187 -0
  96. data/contrib/zstd/lib/libzstd.pc.in +15 -0
  97. data/contrib/zstd/lib/zstd.h +2090 -0
  98. data/ext/depend +2 -0
  99. data/ext/extconf.rb +18 -5
  100. data/ext/extzstd.c +296 -214
  101. data/ext/extzstd.h +81 -36
  102. data/ext/extzstd_nogvls.h +0 -117
  103. data/ext/extzstd_stream.c +622 -0
  104. data/ext/libzstd_conf.h +8 -0
  105. data/ext/zstd_common.c +11 -0
  106. data/ext/zstd_compress.c +15 -0
  107. data/ext/zstd_decompress.c +6 -0
  108. data/ext/zstd_dictbuilder.c +10 -0
  109. data/ext/zstd_dictbuilder_fastcover.c +3 -0
  110. data/ext/zstd_legacy_v01.c +3 -1
  111. data/ext/zstd_legacy_v02.c +3 -1
  112. data/ext/zstd_legacy_v03.c +3 -1
  113. data/ext/zstd_legacy_v04.c +3 -1
  114. data/ext/zstd_legacy_v05.c +3 -1
  115. data/ext/zstd_legacy_v06.c +3 -1
  116. data/ext/zstd_legacy_v07.c +3 -0
  117. data/gemstub.rb +27 -21
  118. data/lib/extzstd.rb +82 -161
  119. data/lib/extzstd/version.rb +1 -1
  120. data/test/test_basic.rb +19 -6
  121. metadata +127 -59
  122. data/contrib/zstd/common/error_private.h +0 -125
  123. data/contrib/zstd/common/error_public.h +0 -77
  124. data/contrib/zstd/common/huf.h +0 -228
  125. data/contrib/zstd/common/zstd.h +0 -475
  126. data/contrib/zstd/common/zstd_common.c +0 -91
  127. data/contrib/zstd/common/zstd_internal.h +0 -238
  128. data/contrib/zstd/compress/huf_compress.c +0 -577
  129. data/contrib/zstd/compress/zbuff_compress.c +0 -327
  130. data/contrib/zstd/compress/zstd_compress.c +0 -3074
  131. data/contrib/zstd/compress/zstd_opt.h +0 -1046
  132. data/contrib/zstd/decompress/huf_decompress.c +0 -894
  133. data/contrib/zstd/decompress/zbuff_decompress.c +0 -294
  134. data/contrib/zstd/decompress/zstd_decompress.c +0 -1362
  135. data/contrib/zstd/dictBuilder/zdict.h +0 -113
  136. data/contrib/zstd/legacy/zstd_legacy.h +0 -140
  137. data/ext/extzstd_buffered.c +0 -265
  138. data/ext/zstd_amalgam.c +0 -18
@@ -0,0 +1,354 @@
1
+ # ################################################################
2
+ # Copyright (c) 2015-2020, Yann Collet, Facebook, Inc.
3
+ # All rights reserved.
4
+ #
5
+ # This source code is licensed under both the BSD-style license (found in the
6
+ # LICENSE file in the root directory of this source tree) and the GPLv2 (found
7
+ # in the COPYING file in the root directory of this source tree).
8
+ # You may select, at your option, one of the above-listed licenses.
9
+ # ################################################################
10
+
11
+ Q = $(if $(filter 1,$(V) $(VERBOSE)),,@)
12
+
13
+ # When cross-compiling from linux to windows, you might
14
+ # need to specify this as "Windows." Fedora build fails
15
+ # without it.
16
+ #
17
+ # Note: mingw-w64 build from linux to windows does not
18
+ # fail on other tested distros (ubuntu, debian) even
19
+ # without manually specifying the TARGET_SYSTEM.
20
+ TARGET_SYSTEM ?= $(OS)
21
+
22
+ # Version numbers
23
+ LIBVER_MAJOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MAJOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./zstd.h`
24
+ LIBVER_MINOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MINOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./zstd.h`
25
+ LIBVER_PATCH_SCRIPT:=`sed -n '/define ZSTD_VERSION_RELEASE/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./zstd.h`
26
+ LIBVER_SCRIPT:= $(LIBVER_MAJOR_SCRIPT).$(LIBVER_MINOR_SCRIPT).$(LIBVER_PATCH_SCRIPT)
27
+ LIBVER_MAJOR := $(shell echo $(LIBVER_MAJOR_SCRIPT))
28
+ LIBVER_MINOR := $(shell echo $(LIBVER_MINOR_SCRIPT))
29
+ LIBVER_PATCH := $(shell echo $(LIBVER_PATCH_SCRIPT))
30
+ LIBVER := $(shell echo $(LIBVER_SCRIPT))
31
+ VERSION?= $(LIBVER)
32
+ CCVER := $(shell $(CC) --version)
33
+
34
+ CPPFLAGS+= -DXXH_NAMESPACE=ZSTD_
35
+ ifeq ($(TARGET_SYSTEM),Windows_NT) # MinGW assumed
36
+ CPPFLAGS += -D__USE_MINGW_ANSI_STDIO # compatibility with %zu formatting
37
+ endif
38
+ DEBUGFLAGS= -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow \
39
+ -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement \
40
+ -Wstrict-prototypes -Wundef -Wpointer-arith \
41
+ -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings \
42
+ -Wredundant-decls -Wmissing-prototypes -Wc++-compat
43
+ CFLAGS += $(DEBUGFLAGS) $(MOREFLAGS)
44
+ FLAGS = $(CPPFLAGS) $(CFLAGS)
45
+
46
+ HAVE_COLORNEVER = $(shell echo a | grep --color=never a > /dev/null 2> /dev/null && echo 1 || echo 0)
47
+ GREP_OPTIONS ?=
48
+ ifeq ($HAVE_COLORNEVER, 1)
49
+ GREP_OPTIONS += --color=never
50
+ endif
51
+ GREP = grep $(GREP_OPTIONS)
52
+
53
+ ZSTDCOMMON_FILES := $(sort $(wildcard common/*.c))
54
+ ZSTDCOMP_FILES := $(sort $(wildcard compress/*.c))
55
+ ZSTDDECOMP_FILES := $(sort $(wildcard decompress/*.c))
56
+ ZDICT_FILES := $(sort $(wildcard dictBuilder/*.c))
57
+ ZDEPR_FILES := $(sort $(wildcard deprecated/*.c))
58
+ ZSTD_FILES := $(ZSTDCOMMON_FILES)
59
+
60
+ ifeq ($(findstring GCC,$(CCVER)),GCC)
61
+ decompress/zstd_decompress_block.o : CFLAGS+=-fno-tree-vectorize
62
+ endif
63
+
64
+ # This is a helper variable that configures a bunch of other variables to new,
65
+ # space-optimized defaults.
66
+ ZSTD_LIB_MINIFY ?= 0
67
+ ifneq ($(ZSTD_LIB_MINIFY), 0)
68
+ HAVE_CC_OZ ?= $(shell echo "" | $(CC) -Oz -x c -c - -o /dev/null 2> /dev/null && echo 1 || echo 0)
69
+ ZSTD_LEGACY_SUPPORT ?= 0
70
+ ZSTD_LIB_DEPRECATED ?= 0
71
+ HUF_FORCE_DECOMPRESS_X1 ?= 1
72
+ ZSTD_FORCE_DECOMPRESS_SHORT ?= 1
73
+ ZSTD_NO_INLINE ?= 1
74
+ ZSTD_STRIP_ERROR_STRINGS ?= 1
75
+ ifneq ($(HAVE_CC_OZ), 0)
76
+ # Some compilers (clang) support an even more space-optimized setting.
77
+ CFLAGS += -Oz
78
+ else
79
+ CFLAGS += -Os
80
+ endif
81
+ CFLAGS += -fno-stack-protector -fomit-frame-pointer -fno-ident \
82
+ -DDYNAMIC_BMI2=0 -DNDEBUG
83
+ else
84
+ CFLAGS += -O3
85
+ endif
86
+
87
+ # Modules
88
+ ZSTD_LIB_COMPRESSION ?= 1
89
+ ZSTD_LIB_DECOMPRESSION ?= 1
90
+ ZSTD_LIB_DICTBUILDER ?= 1
91
+ ZSTD_LIB_DEPRECATED ?= 1
92
+
93
+ # Legacy support
94
+ ZSTD_LEGACY_SUPPORT ?= 5
95
+ ZSTD_LEGACY_MULTITHREADED_API ?= 0
96
+
97
+ # Build size optimizations
98
+ HUF_FORCE_DECOMPRESS_X1 ?= 0
99
+ HUF_FORCE_DECOMPRESS_X2 ?= 0
100
+ ZSTD_FORCE_DECOMPRESS_SHORT ?= 0
101
+ ZSTD_FORCE_DECOMPRESS_LONG ?= 0
102
+ ZSTD_NO_INLINE ?= 0
103
+ ZSTD_STRIP_ERROR_STRINGS ?= 0
104
+
105
+ ifeq ($(ZSTD_LIB_COMPRESSION), 0)
106
+ ZSTD_LIB_DICTBUILDER = 0
107
+ ZSTD_LIB_DEPRECATED = 0
108
+ endif
109
+
110
+ ifeq ($(ZSTD_LIB_DECOMPRESSION), 0)
111
+ ZSTD_LEGACY_SUPPORT = 0
112
+ ZSTD_LIB_DEPRECATED = 0
113
+ endif
114
+
115
+ ifneq ($(ZSTD_LIB_COMPRESSION), 0)
116
+ ZSTD_FILES += $(ZSTDCOMP_FILES)
117
+ endif
118
+
119
+ ifneq ($(ZSTD_LIB_DECOMPRESSION), 0)
120
+ ZSTD_FILES += $(ZSTDDECOMP_FILES)
121
+ endif
122
+
123
+ ifneq ($(ZSTD_LIB_DEPRECATED), 0)
124
+ ZSTD_FILES += $(ZDEPR_FILES)
125
+ endif
126
+
127
+ ifneq ($(ZSTD_LIB_DICTBUILDER), 0)
128
+ ZSTD_FILES += $(ZDICT_FILES)
129
+ endif
130
+
131
+ ifneq ($(HUF_FORCE_DECOMPRESS_X1), 0)
132
+ CFLAGS += -DHUF_FORCE_DECOMPRESS_X1
133
+ endif
134
+
135
+ ifneq ($(HUF_FORCE_DECOMPRESS_X2), 0)
136
+ CFLAGS += -DHUF_FORCE_DECOMPRESS_X2
137
+ endif
138
+
139
+ ifneq ($(ZSTD_FORCE_DECOMPRESS_SHORT), 0)
140
+ CFLAGS += -DZSTD_FORCE_DECOMPRESS_SHORT
141
+ endif
142
+
143
+ ifneq ($(ZSTD_FORCE_DECOMPRESS_LONG), 0)
144
+ CFLAGS += -DZSTD_FORCE_DECOMPRESS_LONG
145
+ endif
146
+
147
+ ifneq ($(ZSTD_NO_INLINE), 0)
148
+ CFLAGS += -DZSTD_NO_INLINE
149
+ endif
150
+
151
+ ifneq ($(ZSTD_STRIP_ERROR_STRINGS), 0)
152
+ CFLAGS += -DZSTD_STRIP_ERROR_STRINGS
153
+ endif
154
+
155
+ ifneq ($(ZSTD_LEGACY_MULTITHREADED_API), 0)
156
+ CFLAGS += -DZSTD_LEGACY_MULTITHREADED_API
157
+ endif
158
+
159
+ ifneq ($(ZSTD_LEGACY_SUPPORT), 0)
160
+ ifeq ($(shell test $(ZSTD_LEGACY_SUPPORT) -lt 8; echo $$?), 0)
161
+ ZSTD_FILES += $(shell ls legacy/*.c | $(GREP) 'v0[$(ZSTD_LEGACY_SUPPORT)-7]')
162
+ endif
163
+ endif
164
+ CPPFLAGS += -DZSTD_LEGACY_SUPPORT=$(ZSTD_LEGACY_SUPPORT)
165
+
166
+ ZSTD_OBJ := $(patsubst %.c,%.o,$(ZSTD_FILES))
167
+
168
+ # macOS linker doesn't support -soname, and use different extension
169
+ # see : https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html
170
+ ifeq ($(shell uname), Darwin)
171
+ SHARED_EXT = dylib
172
+ SHARED_EXT_MAJOR = $(LIBVER_MAJOR).$(SHARED_EXT)
173
+ SHARED_EXT_VER = $(LIBVER).$(SHARED_EXT)
174
+ SONAME_FLAGS = -install_name $(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR) -compatibility_version $(LIBVER_MAJOR) -current_version $(LIBVER)
175
+ else
176
+ SONAME_FLAGS = -Wl,-soname=libzstd.$(SHARED_EXT).$(LIBVER_MAJOR)
177
+ SHARED_EXT = so
178
+ SHARED_EXT_MAJOR = $(SHARED_EXT).$(LIBVER_MAJOR)
179
+ SHARED_EXT_VER = $(SHARED_EXT).$(LIBVER)
180
+ endif
181
+
182
+
183
+ .PHONY: default lib-all all clean install uninstall
184
+
185
+ default: lib-release
186
+
187
+ # alias
188
+ lib-all: all
189
+
190
+ all: lib
191
+
192
+ libzstd.a: ARFLAGS = rcs
193
+ libzstd.a: $(ZSTD_OBJ)
194
+ @echo compiling static library
195
+ $(Q)$(AR) $(ARFLAGS) $@ $^
196
+
197
+ ifneq (,$(filter Windows%,$(TARGET_SYSTEM)))
198
+
199
+ LIBZSTD = dll\libzstd.dll
200
+ $(LIBZSTD): $(ZSTD_FILES)
201
+ @echo compiling dynamic library $(LIBVER)
202
+ $(CC) $(FLAGS) -DZSTD_DLL_EXPORT=1 -Wl,--out-implib,dll\libzstd.dll.a -shared $^ -o $@
203
+
204
+ else
205
+
206
+ LIBZSTD = libzstd.$(SHARED_EXT_VER)
207
+ $(LIBZSTD): LDFLAGS += -shared -fPIC -fvisibility=hidden
208
+ $(LIBZSTD): $(ZSTD_FILES)
209
+ @echo compiling dynamic library $(LIBVER)
210
+ $(Q)$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@
211
+ @echo creating versioned links
212
+ $(Q)ln -sf $@ libzstd.$(SHARED_EXT_MAJOR)
213
+ $(Q)ln -sf $@ libzstd.$(SHARED_EXT)
214
+
215
+ endif
216
+
217
+ .PHONY: libzstd
218
+ libzstd : $(LIBZSTD)
219
+
220
+ .PHONY: lib
221
+ lib : libzstd.a libzstd
222
+
223
+ .PHONY: lib-mt
224
+ %-mt : CPPFLAGS += -DZSTD_MULTITHREAD
225
+ %-mt : LDFLAGS += -pthread
226
+ %-mt : %
227
+ @echo multi-threading build completed
228
+
229
+ .PHONY: lib-release
230
+ %-release : DEBUGFLAGS :=
231
+ %-release : %
232
+ @echo release build completed
233
+
234
+
235
+ # Special case : building library in single-thread mode _and_ without zstdmt_compress.c
236
+ ZSTDMT_FILES = compress/zstdmt_compress.c
237
+ ZSTD_NOMT_FILES = $(filter-out $(ZSTDMT_FILES),$(ZSTD_FILES))
238
+ libzstd-nomt: LDFLAGS += -shared -fPIC -fvisibility=hidden
239
+ libzstd-nomt: $(ZSTD_NOMT_FILES)
240
+ @echo compiling single-thread dynamic library $(LIBVER)
241
+ @echo files : $(ZSTD_NOMT_FILES)
242
+ $(Q)$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@
243
+
244
+ clean:
245
+ $(Q)$(RM) -r *.dSYM # macOS-specific
246
+ $(Q)$(RM) core *.o *.a *.gcda *.$(SHARED_EXT) *.$(SHARED_EXT).* libzstd.pc
247
+ $(Q)$(RM) dll/libzstd.dll dll/libzstd.lib libzstd-nomt*
248
+ $(Q)$(RM) common/*.o compress/*.o decompress/*.o dictBuilder/*.o legacy/*.o deprecated/*.o
249
+ @echo Cleaning library completed
250
+
251
+ #-----------------------------------------------------------------------------
252
+ # make install is validated only for below listed environments
253
+ #-----------------------------------------------------------------------------
254
+ ifneq (,$(filter $(shell uname),Linux Darwin GNU/kFreeBSD GNU OpenBSD FreeBSD NetBSD DragonFly SunOS Haiku))
255
+
256
+ all: libzstd.pc
257
+
258
+ DESTDIR ?=
259
+ # directory variables : GNU conventions prefer lowercase
260
+ # see https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html
261
+ # support both lower and uppercase (BSD), use uppercase in script
262
+ prefix ?= /usr/local
263
+ PREFIX ?= $(prefix)
264
+ exec_prefix ?= $(PREFIX)
265
+ EXEC_PREFIX ?= $(exec_prefix)
266
+ libdir ?= $(EXEC_PREFIX)/lib
267
+ LIBDIR ?= $(libdir)
268
+ includedir ?= $(PREFIX)/include
269
+ INCLUDEDIR ?= $(includedir)
270
+
271
+ PCLIBDIR ?= $(shell echo "$(LIBDIR)" | sed -n -E -e "s@^$(EXEC_PREFIX)(/|$$)@@p")
272
+ PCINCDIR ?= $(shell echo "$(INCLUDEDIR)" | sed -n -E -e "s@^$(PREFIX)(/|$$)@@p")
273
+
274
+ ifeq (,$(PCLIBDIR))
275
+ # Additional prefix check is required, since the empty string is technically a
276
+ # valid PCLIBDIR
277
+ ifeq (,$(shell echo "$(LIBDIR)" | sed -n -E -e "\\@^$(EXEC_PREFIX)(/|$$)@ p"))
278
+ $(error configured libdir ($(LIBDIR)) is outside of prefix ($(PREFIX)), can't generate pkg-config file)
279
+ endif
280
+ endif
281
+
282
+ ifeq (,$(PCINCDIR))
283
+ # Additional prefix check is required, since the empty string is technically a
284
+ # valid PCINCDIR
285
+ ifeq (,$(shell echo "$(INCLUDEDIR)" | sed -n -E -e "\\@^$(PREFIX)(/|$$)@ p"))
286
+ $(error configured includedir ($(INCLUDEDIR)) is outside of exec_prefix ($(EXEC_PREFIX)), can't generate pkg-config file)
287
+ endif
288
+ endif
289
+
290
+ ifneq (,$(filter $(shell uname),FreeBSD NetBSD DragonFly))
291
+ PKGCONFIGDIR ?= $(PREFIX)/libdata/pkgconfig
292
+ else
293
+ PKGCONFIGDIR ?= $(LIBDIR)/pkgconfig
294
+ endif
295
+
296
+ ifneq (,$(filter $(shell uname),SunOS))
297
+ INSTALL ?= ginstall
298
+ else
299
+ INSTALL ?= install
300
+ endif
301
+
302
+ INSTALL_PROGRAM ?= $(INSTALL)
303
+ INSTALL_DATA ?= $(INSTALL) -m 644
304
+
305
+
306
+ libzstd.pc:
307
+ libzstd.pc: libzstd.pc.in
308
+ @echo creating pkgconfig
309
+ $(Q)@sed -E -e 's|@PREFIX@|$(PREFIX)|' \
310
+ -e 's|@LIBDIR@|$(PCLIBDIR)|' \
311
+ -e 's|@INCLUDEDIR@|$(PCINCDIR)|' \
312
+ -e 's|@VERSION@|$(VERSION)|' \
313
+ $< >$@
314
+
315
+ install: install-pc install-static install-shared install-includes
316
+ @echo zstd static and shared library installed
317
+
318
+ install-pc: libzstd.pc
319
+ $(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(PKGCONFIGDIR)/
320
+ $(Q)$(INSTALL_DATA) libzstd.pc $(DESTDIR)$(PKGCONFIGDIR)/
321
+
322
+ install-static: libzstd.a
323
+ @echo Installing static library
324
+ $(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/
325
+ $(Q)$(INSTALL_DATA) libzstd.a $(DESTDIR)$(LIBDIR)
326
+
327
+ install-shared: libzstd
328
+ @echo Installing shared library
329
+ $(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/
330
+ $(Q)$(INSTALL_PROGRAM) $(LIBZSTD) $(DESTDIR)$(LIBDIR)
331
+ $(Q)ln -sf $(LIBZSTD) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR)
332
+ $(Q)ln -sf $(LIBZSTD) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT)
333
+
334
+ install-includes:
335
+ @echo Installing includes
336
+ $(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(INCLUDEDIR)/
337
+ $(Q)$(INSTALL_DATA) zstd.h $(DESTDIR)$(INCLUDEDIR)
338
+ $(Q)$(INSTALL_DATA) common/zstd_errors.h $(DESTDIR)$(INCLUDEDIR)
339
+ $(Q)$(INSTALL_DATA) deprecated/zbuff.h $(DESTDIR)$(INCLUDEDIR) # prototypes generate deprecation warnings
340
+ $(Q)$(INSTALL_DATA) dictBuilder/zdict.h $(DESTDIR)$(INCLUDEDIR)
341
+
342
+ uninstall:
343
+ $(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.a
344
+ $(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT)
345
+ $(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR)
346
+ $(Q)$(RM) $(DESTDIR)$(LIBDIR)/$(LIBZSTD)
347
+ $(Q)$(RM) $(DESTDIR)$(PKGCONFIGDIR)/libzstd.pc
348
+ $(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zstd.h
349
+ $(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zstd_errors.h
350
+ $(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zbuff.h # Deprecated streaming functions
351
+ $(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zdict.h
352
+ @echo zstd libraries successfully uninstalled
353
+
354
+ endif
@@ -0,0 +1,179 @@
1
+ Zstandard library files
2
+ ================================
3
+
4
+ The __lib__ directory is split into several sub-directories,
5
+ in order to make it easier to select or exclude features.
6
+
7
+
8
+ #### Building
9
+
10
+ `Makefile` script is provided, supporting [Makefile conventions](https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html#Makefile-Conventions),
11
+ including commands variables, staged install, directory variables and standard targets.
12
+ - `make` : generates both static and dynamic libraries
13
+ - `make install` : install libraries and headers in target system directories
14
+
15
+ `libzstd` default scope is pretty large, including compression, decompression, dictionary builder,
16
+ and support for decoding legacy formats >= v0.5.0.
17
+ The scope can be reduced on demand (see paragraph _modular build_).
18
+
19
+
20
+ #### Multithreading support
21
+
22
+ Multithreading is disabled by default when building with `make`.
23
+ Enabling multithreading requires 2 conditions :
24
+ - set build macro `ZSTD_MULTITHREAD` (`-DZSTD_MULTITHREAD` for `gcc`)
25
+ - for POSIX systems : compile with pthread (`-pthread` compilation flag for `gcc`)
26
+
27
+ Both conditions are automatically applied when invoking `make lib-mt` target.
28
+
29
+ When linking a POSIX program with a multithreaded version of `libzstd`,
30
+ note that it's necessary to invoke the `-pthread` flag during link stage.
31
+
32
+ Multithreading capabilities are exposed
33
+ via the [advanced API defined in `lib/zstd.h`](https://github.com/facebook/zstd/blob/v1.4.3/lib/zstd.h#L351).
34
+
35
+
36
+ #### API
37
+
38
+ Zstandard's stable API is exposed within [lib/zstd.h](zstd.h).
39
+
40
+
41
+ #### Advanced API
42
+
43
+ Optional advanced features are exposed via :
44
+
45
+ - `lib/common/zstd_errors.h` : translates `size_t` function results
46
+ into a `ZSTD_ErrorCode`, for accurate error handling.
47
+
48
+ - `ZSTD_STATIC_LINKING_ONLY` : if this macro is defined _before_ including `zstd.h`,
49
+ it unlocks access to the experimental API,
50
+ exposed in the second part of `zstd.h`.
51
+ All definitions in the experimental APIs are unstable,
52
+ they may still change in the future, or even be removed.
53
+ As a consequence, experimental definitions shall ___never be used with dynamic library___ !
54
+ Only static linking is allowed.
55
+
56
+
57
+ #### Modular build
58
+
59
+ It's possible to compile only a limited set of features within `libzstd`.
60
+ The file structure is designed to make this selection manually achievable for any build system :
61
+
62
+ - Directory `lib/common` is always required, for all variants.
63
+
64
+ - Compression source code lies in `lib/compress`
65
+
66
+ - Decompression source code lies in `lib/decompress`
67
+
68
+ - It's possible to include only `compress` or only `decompress`, they don't depend on each other.
69
+
70
+ - `lib/dictBuilder` : makes it possible to generate dictionaries from a set of samples.
71
+ The API is exposed in `lib/dictBuilder/zdict.h`.
72
+ This module depends on both `lib/common` and `lib/compress` .
73
+
74
+ - `lib/legacy` : makes it possible to decompress legacy zstd formats, starting from `v0.1.0`.
75
+ This module depends on `lib/common` and `lib/decompress`.
76
+ To enable this feature, define `ZSTD_LEGACY_SUPPORT` during compilation.
77
+ Specifying a number limits versions supported to that version onward.
78
+ For example, `ZSTD_LEGACY_SUPPORT=2` means : "support legacy formats >= v0.2.0".
79
+ Conversely, `ZSTD_LEGACY_SUPPORT=0` means "do __not__ support legacy formats".
80
+ By default, this build macro is set as `ZSTD_LEGACY_SUPPORT=5`.
81
+ Decoding supported legacy format is a transparent capability triggered within decompression functions.
82
+ It's also allowed to invoke legacy API directly, exposed in `lib/legacy/zstd_legacy.h`.
83
+ Each version does also provide its own set of advanced API.
84
+ For example, advanced API for version `v0.4` is exposed in `lib/legacy/zstd_v04.h` .
85
+
86
+ - While invoking `make libzstd`, it's possible to define build macros
87
+ `ZSTD_LIB_COMPRESSION, ZSTD_LIB_DECOMPRESSION`, `ZSTD_LIB_DICTBUILDER`,
88
+ and `ZSTD_LIB_DEPRECATED` as `0` to forgo compilation of the
89
+ corresponding features. This will also disable compilation of all
90
+ dependencies (eg. `ZSTD_LIB_COMPRESSION=0` will also disable
91
+ dictBuilder).
92
+
93
+ - There are a number of options that can help minimize the binary size of
94
+ `libzstd`.
95
+
96
+ The first step is to select the components needed (using the above-described
97
+ `ZSTD_LIB_COMPRESSION` etc.).
98
+
99
+ The next step is to set `ZSTD_LIB_MINIFY` to `1` when invoking `make`. This
100
+ disables various optional components and changes the compilation flags to
101
+ prioritize space-saving.
102
+
103
+ Detailed options: Zstandard's code and build environment is set up by default
104
+ to optimize above all else for performance. In pursuit of this goal, Zstandard
105
+ makes significant trade-offs in code size. For example, Zstandard often has
106
+ more than one implementation of a particular component, with each
107
+ implementation optimized for different scenarios. For example, the Huffman
108
+ decoder has complementary implementations that decode the stream one symbol at
109
+ a time or two symbols at a time. Zstd normally includes both (and dispatches
110
+ between them at runtime), but by defining `HUF_FORCE_DECOMPRESS_X1` or
111
+ `HUF_FORCE_DECOMPRESS_X2`, you can force the use of one or the other, avoiding
112
+ compilation of the other. Similarly, `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT`
113
+ and `ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG` force the compilation and use of
114
+ only one or the other of two decompression implementations. The smallest
115
+ binary is achieved by using `HUF_FORCE_DECOMPRESS_X1` and
116
+ `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT` (implied by `ZSTD_LIB_MINIFY`).
117
+
118
+ For squeezing the last ounce of size out, you can also define
119
+ `ZSTD_NO_INLINE`, which disables inlining, and `ZSTD_STRIP_ERROR_STRINGS`,
120
+ which removes the error messages that are otherwise returned by
121
+ `ZSTD_getErrorName` (implied by `ZSTD_LIB_MINIFY`).
122
+
123
+ Finally, when integrating into your application, make sure you're doing link-
124
+ time optimation and unused symbol garbage collection (via some combination of,
125
+ e.g., `-flto`, `-ffat-lto-objects`, `-fuse-linker-plugin`,
126
+ `-ffunction-sections`, `-fdata-sections`, `-fmerge-all-constants`,
127
+ `-Wl,--gc-sections`, `-Wl,-z,norelro`, and an archiver that understands
128
+ the compiler's intermediate representation, e.g., `AR=gcc-ar`). Consult your
129
+ compiler's documentation.
130
+
131
+ - While invoking `make libzstd`, the build macro `ZSTD_LEGACY_MULTITHREADED_API=1`
132
+ will expose the deprecated `ZSTDMT` API exposed by `zstdmt_compress.h` in
133
+ the shared library, which is now hidden by default.
134
+
135
+ - The build macro `DYNAMIC_BMI2` can be set to 1 or 0 in order to generate binaries
136
+ which can detect at runtime the presence of BMI2 instructions, and use them only if present.
137
+ These instructions contribute to better performance, notably on the decoder side.
138
+ By default, this feature is automatically enabled on detecting
139
+ the right instruction set (x64) and compiler (clang or gcc >= 5).
140
+ It's obviously disabled for different cpus,
141
+ or when BMI2 instruction set is _required_ by the compiler command line
142
+ (in this case, only the BMI2 code path is generated).
143
+ Setting this macro will either force to generate the BMI2 dispatcher (1)
144
+ or prevent it (0). It overrides automatic detection.
145
+
146
+
147
+ #### Windows : using MinGW+MSYS to create DLL
148
+
149
+ DLL can be created using MinGW+MSYS with the `make libzstd` command.
150
+ This command creates `dll\libzstd.dll` and the import library `dll\libzstd.lib`.
151
+ The import library is only required with Visual C++.
152
+ The header file `zstd.h` and the dynamic library `dll\libzstd.dll` are required to
153
+ compile a project using gcc/MinGW.
154
+ The dynamic library has to be added to linking options.
155
+ It means that if a project that uses ZSTD consists of a single `test-dll.c`
156
+ file it should be linked with `dll\libzstd.dll`. For example:
157
+ ```
158
+ gcc $(CFLAGS) -Iinclude/ test-dll.c -o test-dll dll\libzstd.dll
159
+ ```
160
+ The compiled executable will require ZSTD DLL which is available at `dll\libzstd.dll`.
161
+
162
+
163
+ #### Deprecated API
164
+
165
+ Obsolete API on their way out are stored in directory `lib/deprecated`.
166
+ At this stage, it contains older streaming prototypes, in `lib/deprecated/zbuff.h`.
167
+ These prototypes will be removed in some future version.
168
+ Consider migrating code towards supported streaming API exposed in `zstd.h`.
169
+
170
+
171
+ #### Miscellaneous
172
+
173
+ The other files are not source code. There are :
174
+
175
+ - `BUCK` : support for `buck` build system (https://buckbuild.com/)
176
+ - `Makefile` : `make` script to build and install zstd library (static and dynamic)
177
+ - `README.md` : this file
178
+ - `dll/` : resources directory for Windows compilation
179
+ - `libzstd.pc.in` : script for `pkg-config` (used in `make install`)