nutcracker 0.4.0.16 → 0.4.1.18
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Rakefile +3 -3
- data/ext/nutcracker/ChangeLog +33 -8
- data/ext/nutcracker/Makefile.in +34 -21
- data/ext/nutcracker/README.md +61 -26
- data/ext/nutcracker/aclocal.m4 +34 -31
- data/ext/nutcracker/autom4te.cache/output.0 +1875 -1330
- data/ext/nutcracker/autom4te.cache/output.1 +1875 -1330
- data/ext/nutcracker/autom4te.cache/requests +232 -451
- data/ext/nutcracker/autom4te.cache/traces.0 +2256 -2129
- data/ext/nutcracker/autom4te.cache/traces.1 +73 -59
- data/ext/nutcracker/config.h.in +1 -2
- data/ext/nutcracker/config.h.in~ +333 -0
- data/ext/nutcracker/config/compile +1 -1
- data/ext/nutcracker/config/config.guess +13 -160
- data/ext/nutcracker/config/config.sub +25 -11
- data/ext/nutcracker/config/depcomp +1 -1
- data/ext/nutcracker/config/install-sh +170 -196
- data/ext/nutcracker/config/ltmain.sh +3509 -2018
- data/ext/nutcracker/config/missing +1 -1
- data/ext/nutcracker/configure +1874 -1329
- data/ext/nutcracker/configure.ac +3 -2
- data/ext/nutcracker/contrib/Makefile.in +18 -5
- data/ext/nutcracker/contrib/yaml-0.1.4/LICENSE +19 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/Makefile.am +20 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/Makefile.in +832 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/README +27 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/aclocal.m4 +1157 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/output.0 +13342 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/output.1 +14611 -0
- data/ext/nutcracker/{autom4te.cache → contrib/yaml-0.1.4/autom4te.cache}/output.2 +3465 -8761
- data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/requests +516 -0
- data/ext/nutcracker/{autom4te.cache/traces.2 → contrib/yaml-0.1.4/autom4te.cache/traces.0} +662 -698
- data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/traces.1 +577 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/traces.2 +2721 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config.h.in +79 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config.h.in~ +80 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config/compile +347 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config/config.guess +1421 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config/config.sub +1807 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config/depcomp +791 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config/install-sh +501 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config/ltmain.sh +11147 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config/missing +215 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/config/test-driver +148 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/configure +14611 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/configure.ac +75 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/doc/doxygen.cfg +222 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/include/yaml.h +1971 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/m4/libtool.m4 +8369 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/m4/ltoptions.m4 +437 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/m4/ltsugar.m4 +124 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/m4/ltversion.m4 +23 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/m4/lt~obsolete.m4 +99 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/Makefile.am +4 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/Makefile.in +600 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/api.c +1392 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/dumper.c +394 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/emitter.c +2329 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/loader.c +432 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/parser.c +1374 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/reader.c +465 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/scanner.c +3570 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/writer.c +141 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/src/yaml_private.h +640 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/Makefile.am +8 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/Makefile.in +1083 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/example-deconstructor-alt.c +800 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/example-deconstructor.c +1130 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/example-reformatter-alt.c +217 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/example-reformatter.c +202 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-dumper.c +311 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-emitter.c +327 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-loader.c +63 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-parser.c +63 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-scanner.c +63 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/test-reader.c +354 -0
- data/ext/nutcracker/contrib/yaml-0.1.4/tests/test-version.c +29 -0
- data/ext/nutcracker/m4/libtool.m4 +1474 -1087
- data/ext/nutcracker/m4/ltoptions.m4 +90 -37
- data/ext/nutcracker/m4/ltsugar.m4 +4 -3
- data/ext/nutcracker/m4/ltversion.m4 +6 -6
- data/ext/nutcracker/m4/lt~obsolete.m4 +4 -3
- data/ext/nutcracker/man/nutcracker.8 +1 -1
- data/ext/nutcracker/notes/memcache.md +162 -0
- data/ext/nutcracker/notes/recommendation.md +10 -5
- data/ext/nutcracker/notes/redis.md +23 -9
- data/ext/nutcracker/scripts/nutcracker.init +10 -0
- data/ext/nutcracker/scripts/nutcracker.init.debian +83 -0
- data/ext/nutcracker/scripts/nutcracker.spec +36 -2
- data/ext/nutcracker/scripts/redis-check.sh +4 -0
- data/ext/nutcracker/src/Makefile.am +6 -1
- data/ext/nutcracker/src/Makefile.in +26 -12
- data/ext/nutcracker/src/event/Makefile.in +19 -6
- data/ext/nutcracker/src/hashkit/Makefile.in +19 -6
- data/ext/nutcracker/src/hashkit/nc_jenkins.c +1 -1
- data/ext/nutcracker/src/hashkit/nc_ketama.c +3 -3
- data/ext/nutcracker/src/nc.c +1 -1
- data/ext/nutcracker/src/nc_conf.c +67 -19
- data/ext/nutcracker/src/nc_conf.h +9 -4
- data/ext/nutcracker/src/nc_connection.c +35 -2
- data/ext/nutcracker/src/nc_connection.h +53 -47
- data/ext/nutcracker/src/nc_core.c +8 -1
- data/ext/nutcracker/src/nc_message.c +23 -7
- data/ext/nutcracker/src/nc_message.h +24 -1
- data/ext/nutcracker/src/nc_proxy.c +14 -3
- data/ext/nutcracker/src/nc_rbtree.c +1 -5
- data/ext/nutcracker/src/nc_request.c +58 -10
- data/ext/nutcracker/src/nc_response.c +27 -4
- data/ext/nutcracker/src/nc_server.c +33 -5
- data/ext/nutcracker/src/nc_server.h +10 -9
- data/ext/nutcracker/src/nc_string.h +17 -0
- data/ext/nutcracker/src/nc_util.c +5 -1
- data/ext/nutcracker/src/proto/Makefile.in +19 -6
- data/ext/nutcracker/src/proto/nc_memcache.c +76 -12
- data/ext/nutcracker/src/proto/nc_proto.h +9 -0
- data/ext/nutcracker/src/proto/nc_redis.c +400 -18
- data/lib/nutcracker.rb +1 -1
- data/lib/nutcracker/version.rb +1 -1
- metadata +61 -6
- data/ext/nutcracker/notes/memcache.txt +0 -123
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 96ee5eec694e7984421bb7cc24f4cb081ffebea3
|
4
|
+
data.tar.gz: 357729667f8ea82d7c6b39f14ada570040f675c9
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 22cd6aea9938befd72afb512e5b0d003795d29760618c39c7d76b1e433f3dcafec84a59063dd5d1b820cb36758ded7a335e514ea4202a072714bd6a7a168ae7c
|
7
|
+
data.tar.gz: a99b46cb63b4b447a5ba588317fc0bf7bc411a976ae7f72542b304b521a5b108bdb63f8f5a6fb4f644a6ff54cc47debbd847c70b37b12b0115323a69c3282253
|
data/Rakefile
CHANGED
@@ -10,12 +10,12 @@ sversion = Nutcracker.version.split(".")[0..2].join(".")
|
|
10
10
|
|
11
11
|
desc "Download Nutcracker c app"
|
12
12
|
task :download do
|
13
|
-
"
|
13
|
+
"nutcracker-#{sversion}.tar.gz".tap do |tarball|
|
14
14
|
sh "mkdir ext" unless File.directory? "ext"
|
15
15
|
sh "rm -rf ext/nutcracker"
|
16
|
-
sh "wget https://github.com/twitter/twemproxy/archive/v#{sversion}.tar.gz -O #{tarball}"
|
16
|
+
#sh "wget https://github.com/twitter/twemproxy/archive/v#{sversion}.tar.gz -O #{tarball}"
|
17
17
|
sh "tar -zxvf #{tarball}"
|
18
|
-
sh "mv
|
18
|
+
sh "mv nutcracker-#{sversion} ext/nutcracker"
|
19
19
|
Dir.chdir("ext/nutcracker") do
|
20
20
|
sh "autoreconf -fvi"
|
21
21
|
c = File.read("configure").gsub("-${am__api_version}","")
|
data/ext/nutcracker/ChangeLog
CHANGED
@@ -1,3 +1,36 @@
|
|
1
|
+
2015-22-06 Manju Rajashekhar <manj@cs.stanford.edu>
|
2
|
+
* twemproxy: version 0.4.1 release
|
3
|
+
redis_auth is only valid for a redis pool
|
4
|
+
getaddrinfo returns non-zero +ve value on error
|
5
|
+
fix-hang-when-command-only (charsyam)
|
6
|
+
fix bug crash when get command without key and whitespace (charsyam)
|
7
|
+
mark server as failed on protocol level transiet failures like -OOM, -LOADING, etc
|
8
|
+
implemented support for parsing fine grained redis error response
|
9
|
+
remove redundant conditional judgement in rbtree deletion (leo ma)
|
10
|
+
fix bug mset has invalid pair (charsyam)
|
11
|
+
fix bug mset has invalid pair (charsyam)
|
12
|
+
temp fix a core on kqueue (idning)
|
13
|
+
support "touch" command for memcached (panmiaocai)
|
14
|
+
fix redis parse rsp bug (charsyam)
|
15
|
+
SORT command can take multiple arguments. So it should be part of redis_argn() and not redis_arg0()
|
16
|
+
remove incorrect assert because client could send data after sending a quit request which must be discarded
|
17
|
+
allow file permissions to be set for UNIX domain listening socket (ori liveneh)
|
18
|
+
return error if formatted is greater than mbuf size by using nc_vsnprintf() in msg_prepend_format()
|
19
|
+
fix req_make_reply on msg_get, mark it as response (idning)
|
20
|
+
redis database select upon connect (arne claus)
|
21
|
+
redis_auth (charsyam)
|
22
|
+
allow null key(empty key) (idning)
|
23
|
+
fix core on invalid mset like "mset a a a" (idning)
|
24
|
+
|
25
|
+
2014-18-10 idning <idning@gmail.com>
|
26
|
+
* twemproxy: version 0.4.0 release
|
27
|
+
mget improve (idning)
|
28
|
+
many new commands supported: LEX, PFADD, PFMERGE, SORT, PING, QUIT, SCAN... (mattrobenolt, areina, idning)
|
29
|
+
handle max open file limit(allenlz)
|
30
|
+
add notice-log and use ms time in log(idning)
|
31
|
+
fix bug in string_compare (andyqzb)
|
32
|
+
fix deadlock in sighandler (idning)
|
33
|
+
|
1
34
|
2013-20-12 Manju Rajashekhar <manj@cs.stanford.edu>
|
2
35
|
* twemproxy: version 0.3.0 release
|
3
36
|
SRANDMEMBER support for the optional count argument (mkhq)
|
@@ -28,12 +61,10 @@
|
|
28
61
|
add support for hash tags
|
29
62
|
|
30
63
|
2012-18-10 Manju Rajashekhar <manj@twitter.com>
|
31
|
-
|
32
64
|
* twemproxy: version 0.2.2 release
|
33
65
|
fix the off-by-one error when calculating redis key length
|
34
66
|
|
35
67
|
2012-12-10 Manju Rajashekhar <manj@twitter.com>
|
36
|
-
|
37
68
|
* twemproxy: version 0.2.1 release
|
38
69
|
don't use buf in conf_add_server
|
39
70
|
allow an optional instance name for consistent hashing (charsyam)
|
@@ -41,7 +72,6 @@
|
|
41
72
|
add stats-bind-any -a option (charsyam)
|
42
73
|
|
43
74
|
2012-12-03 Manju Rajashekhar <manj@twitter.com>
|
44
|
-
|
45
75
|
* twemproxy: version 0.2.0 release
|
46
76
|
add -D or --describe-stats command-line argument to print stats description
|
47
77
|
redis support in twemproxy
|
@@ -54,7 +84,6 @@
|
|
54
84
|
delete stats tracking memcache requests and responses; stats module no longer tracks protocol related stats
|
55
85
|
|
56
86
|
2012-10-27 Manju Rajashekhar <manj@twitter.com>
|
57
|
-
|
58
87
|
* twemproxy: version 0.1.20 release
|
59
88
|
on msg_repair, msg->pos should point to nbuf->pos and not nbuf->last
|
60
89
|
refactor memcache parsing code into proto directory
|
@@ -63,23 +92,19 @@
|
|
63
92
|
fix log_hexdump and loga_hexdump
|
64
93
|
|
65
94
|
2012-07-31 Manju Rajashekhar <manj@twitter.com>
|
66
|
-
|
67
95
|
* twemproxy: version 0.1.19 release
|
68
96
|
close server connection on a stray response (yashh, bmatheny)
|
69
97
|
|
70
98
|
2012-06-19 Manju Rajashekhar <manj@twitter.com>
|
71
|
-
|
72
99
|
* twemproxy: version 0.1.18 release
|
73
100
|
command line option to set mbuf chunk size
|
74
101
|
|
75
102
|
2012-05-09 Manju Rajashekhar <manj@twitter.com>
|
76
|
-
|
77
103
|
* twemproxy: version 0.1.17 release
|
78
104
|
use _exit(0) instead of exit(0) when daemonizing
|
79
105
|
use loga instead of log_stderr in nc_stacktrace
|
80
106
|
|
81
107
|
2012-02-09 Manju Rajashekhar <manj@twitter.com>
|
82
|
-
|
83
108
|
* twemproxy: version 0.1.16 release
|
84
109
|
twemproxy (aka nutcracker) is a fast and lightweight proxy for memcached protocol.
|
85
110
|
|
data/ext/nutcracker/Makefile.in
CHANGED
@@ -1,7 +1,7 @@
|
|
1
|
-
# Makefile.in generated by automake 1.
|
1
|
+
# Makefile.in generated by automake 1.15 from Makefile.am.
|
2
2
|
# @configure_input@
|
3
3
|
|
4
|
-
# Copyright (C) 1994-
|
4
|
+
# Copyright (C) 1994-2014 Free Software Foundation, Inc.
|
5
5
|
|
6
6
|
# This Makefile.in is free software; the Free Software Foundation
|
7
7
|
# gives unlimited permission to copy and/or distribute it,
|
@@ -14,7 +14,17 @@
|
|
14
14
|
|
15
15
|
@SET_MAKE@
|
16
16
|
VPATH = @srcdir@
|
17
|
-
am__is_gnu_make =
|
17
|
+
am__is_gnu_make = { \
|
18
|
+
if test -z '$(MAKELEVEL)'; then \
|
19
|
+
false; \
|
20
|
+
elif test -n '$(MAKE_HOST)'; then \
|
21
|
+
true; \
|
22
|
+
elif test -n '$(MAKE_VERSION)' && test -n '$(CURDIR)'; then \
|
23
|
+
true; \
|
24
|
+
else \
|
25
|
+
false; \
|
26
|
+
fi; \
|
27
|
+
}
|
18
28
|
am__make_running_with_option = \
|
19
29
|
case $${target_option-} in \
|
20
30
|
?) ;; \
|
@@ -78,15 +88,6 @@ POST_UNINSTALL = :
|
|
78
88
|
build_triplet = @build@
|
79
89
|
host_triplet = @host@
|
80
90
|
subdir = .
|
81
|
-
DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am \
|
82
|
-
$(top_srcdir)/configure $(am__configure_deps) \
|
83
|
-
$(srcdir)/config.h.in $(dist_man_MANS) ChangeLog \
|
84
|
-
config/compile config/config.guess config/config.sub \
|
85
|
-
config/install-sh config/missing config/ltmain.sh \
|
86
|
-
$(top_srcdir)/config/compile $(top_srcdir)/config/config.guess \
|
87
|
-
$(top_srcdir)/config/config.sub \
|
88
|
-
$(top_srcdir)/config/install-sh $(top_srcdir)/config/ltmain.sh \
|
89
|
-
$(top_srcdir)/config/missing
|
90
91
|
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
|
91
92
|
am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \
|
92
93
|
$(top_srcdir)/m4/ltoptions.m4 $(top_srcdir)/m4/ltsugar.m4 \
|
@@ -94,6 +95,8 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \
|
|
94
95
|
$(top_srcdir)/configure.ac
|
95
96
|
am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
|
96
97
|
$(ACLOCAL_M4)
|
98
|
+
DIST_COMMON = $(srcdir)/Makefile.am $(top_srcdir)/configure \
|
99
|
+
$(am__configure_deps) $(am__DIST_COMMON)
|
97
100
|
am__CONFIG_DISTCLEAN_FILES = config.status config.cache config.log \
|
98
101
|
configure.lineno config.status.lineno
|
99
102
|
mkinstalldirs = $(install_sh) -d
|
@@ -188,6 +191,14 @@ ETAGS = etags
|
|
188
191
|
CTAGS = ctags
|
189
192
|
CSCOPE = cscope
|
190
193
|
DIST_SUBDIRS = $(SUBDIRS)
|
194
|
+
am__DIST_COMMON = $(dist_man_MANS) $(srcdir)/Makefile.in \
|
195
|
+
$(srcdir)/config.h.in $(top_srcdir)/config/compile \
|
196
|
+
$(top_srcdir)/config/config.guess \
|
197
|
+
$(top_srcdir)/config/config.sub \
|
198
|
+
$(top_srcdir)/config/install-sh $(top_srcdir)/config/ltmain.sh \
|
199
|
+
$(top_srcdir)/config/missing ChangeLog config/compile \
|
200
|
+
config/config.guess config/config.sub config/install-sh \
|
201
|
+
config/ltmain.sh config/missing
|
191
202
|
DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST)
|
192
203
|
distdir = $(PACKAGE)-$(VERSION)
|
193
204
|
top_distdir = $(distdir)
|
@@ -273,6 +284,7 @@ LIBTOOL = @LIBTOOL@
|
|
273
284
|
LIPO = @LIPO@
|
274
285
|
LN_S = @LN_S@
|
275
286
|
LTLIBOBJS = @LTLIBOBJS@
|
287
|
+
LT_SYS_LIBRARY_PATH = @LT_SYS_LIBRARY_PATH@
|
276
288
|
MAKEINFO = @MAKEINFO@
|
277
289
|
MANIFEST_TOOL = @MANIFEST_TOOL@
|
278
290
|
MKDIR_P = @MKDIR_P@
|
@@ -374,7 +386,6 @@ $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps)
|
|
374
386
|
echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign Makefile'; \
|
375
387
|
$(am__cd) $(top_srcdir) && \
|
376
388
|
$(AUTOMAKE) --foreign Makefile
|
377
|
-
.PRECIOUS: Makefile
|
378
389
|
Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status
|
379
390
|
@case '$?' in \
|
380
391
|
*config.status*) \
|
@@ -648,15 +659,15 @@ dist-xz: distdir
|
|
648
659
|
$(am__post_remove_distdir)
|
649
660
|
|
650
661
|
dist-tarZ: distdir
|
651
|
-
@echo WARNING: "Support for
|
652
|
-
|
662
|
+
@echo WARNING: "Support for distribution archives compressed with" \
|
663
|
+
"legacy program 'compress' is deprecated." >&2
|
653
664
|
@echo WARNING: "It will be removed altogether in Automake 2.0" >&2
|
654
665
|
tardir=$(distdir) && $(am__tar) | compress -c >$(distdir).tar.Z
|
655
666
|
$(am__post_remove_distdir)
|
656
667
|
|
657
668
|
dist-shar: distdir
|
658
|
-
@echo WARNING: "Support for distribution archives
|
659
|
-
|
669
|
+
@echo WARNING: "Support for shar distribution archives is" \
|
670
|
+
"deprecated." >&2
|
660
671
|
@echo WARNING: "It will be removed altogether in Automake 2.0" >&2
|
661
672
|
shar $(distdir) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).shar.gz
|
662
673
|
$(am__post_remove_distdir)
|
@@ -692,17 +703,17 @@ distcheck: dist
|
|
692
703
|
esac
|
693
704
|
chmod -R a-w $(distdir)
|
694
705
|
chmod u+w $(distdir)
|
695
|
-
mkdir $(distdir)/_build $(distdir)/_inst
|
706
|
+
mkdir $(distdir)/_build $(distdir)/_build/sub $(distdir)/_inst
|
696
707
|
chmod a-w $(distdir)
|
697
708
|
test -d $(distdir)/_build || exit 0; \
|
698
709
|
dc_install_base=`$(am__cd) $(distdir)/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \
|
699
710
|
&& dc_destdir="$${TMPDIR-/tmp}/am-dc-$$$$/" \
|
700
711
|
&& am__cwd=`pwd` \
|
701
|
-
&& $(am__cd) $(distdir)/_build \
|
702
|
-
&&
|
712
|
+
&& $(am__cd) $(distdir)/_build/sub \
|
713
|
+
&& ../../configure \
|
703
714
|
$(AM_DISTCHECK_CONFIGURE_FLAGS) \
|
704
715
|
$(DISTCHECK_CONFIGURE_FLAGS) \
|
705
|
-
--srcdir
|
716
|
+
--srcdir=../.. --prefix="$$dc_install_base" \
|
706
717
|
&& $(MAKE) $(AM_MAKEFLAGS) \
|
707
718
|
&& $(MAKE) $(AM_MAKEFLAGS) dvi \
|
708
719
|
&& $(MAKE) $(AM_MAKEFLAGS) check \
|
@@ -885,6 +896,8 @@ uninstall-man: uninstall-man8
|
|
885
896
|
mostlyclean-libtool pdf pdf-am ps ps-am tags tags-am uninstall \
|
886
897
|
uninstall-am uninstall-man uninstall-man8
|
887
898
|
|
899
|
+
.PRECIOUS: Makefile
|
900
|
+
|
888
901
|
|
889
902
|
# Tell versions [3.59,3.63) of GNU make to not export all variables.
|
890
903
|
# Otherwise a system limit (for SysV at least) may be exceeded.
|
data/ext/nutcracker/README.md
CHANGED
@@ -1,30 +1,37 @@
|
|
1
1
|
# twemproxy (nutcracker) [![Build Status](https://secure.travis-ci.org/twitter/twemproxy.png)](http://travis-ci.org/twitter/twemproxy)
|
2
2
|
|
3
|
-
**twemproxy** (pronounced "two-em-proxy"), aka **nutcracker** is a fast and lightweight proxy for [memcached](http://www.memcached.org/) and [redis](http://redis.io/) protocol. It was primarily
|
3
|
+
**twemproxy** (pronounced "two-em-proxy"), aka **nutcracker** is a fast and lightweight proxy for [memcached](http://www.memcached.org/) and [redis](http://redis.io/) protocol. It was built primarily to reduce the number of connections to the caching servers on the backend. This, together with protocol pipelining and sharding enables you to horizontally scale your distributed caching architecture.
|
4
4
|
|
5
5
|
## Build
|
6
6
|
|
7
|
-
To build
|
7
|
+
To build twemproxy from [distribution tarball](https://drive.google.com/open?id=0B6pVMMV5F5dfMUdJV25abllhUWM&authuser=0):
|
8
8
|
|
9
9
|
$ ./configure
|
10
10
|
$ make
|
11
11
|
$ sudo make install
|
12
12
|
|
13
|
-
To build
|
13
|
+
To build twemproxy from [distribution tarball](https://drive.google.com/open?id=0B6pVMMV5F5dfMUdJV25abllhUWM&authuser=0) in _debug mode_:
|
14
14
|
|
15
15
|
$ CFLAGS="-ggdb3 -O0" ./configure --enable-debug=full
|
16
16
|
$ make
|
17
17
|
$ sudo make install
|
18
18
|
|
19
|
-
To build
|
19
|
+
To build twemproxy from source with _debug logs enabled_ and _assertions enabled_:
|
20
20
|
|
21
21
|
$ git clone git@github.com:twitter/twemproxy.git
|
22
22
|
$ cd twemproxy
|
23
23
|
$ autoreconf -fvi
|
24
|
-
$ ./configure --enable-debug=
|
24
|
+
$ ./configure --enable-debug=full
|
25
25
|
$ make
|
26
26
|
$ src/nutcracker -h
|
27
27
|
|
28
|
+
A quick checklist:
|
29
|
+
|
30
|
+
+ Use newer version of gcc (older version of gcc has problems)
|
31
|
+
+ Use CFLAGS="-O1" ./configure && make
|
32
|
+
+ Use CFLAGS="-O3 -fno-strict-aliasing" ./configure && make
|
33
|
+
+ `autoreconf -fvi && ./configure` needs `automake` and `libtool` to be installed
|
34
|
+
|
28
35
|
## Features
|
29
36
|
|
30
37
|
+ Fast.
|
@@ -35,12 +42,12 @@ To build nutcracker from source with _debug logs enabled_ and _assertions disabl
|
|
35
42
|
+ Supports proxying to multiple servers.
|
36
43
|
+ Supports multiple server pools simultaneously.
|
37
44
|
+ Shard data automatically across multiple servers.
|
38
|
-
+ Implements the complete [memcached ascii](notes/memcache.
|
45
|
+
+ Implements the complete [memcached ascii](notes/memcache.md) and [redis](notes/redis.md) protocol.
|
39
46
|
+ Easy configuration of server pools through a YAML file.
|
40
47
|
+ Supports multiple hashing modes including consistent hashing and distribution.
|
41
48
|
+ Can be configured to disable nodes on failures.
|
42
|
-
+ Observability
|
43
|
-
+ Works with Linux, *BSD, OS X and
|
49
|
+
+ Observability via stats exposed on the stats monitoring port.
|
50
|
+
+ Works with Linux, *BSD, OS X and SmartOS (Solaris)
|
44
51
|
|
45
52
|
## Help
|
46
53
|
|
@@ -54,7 +61,7 @@ To build nutcracker from source with _debug logs enabled_ and _assertions disabl
|
|
54
61
|
-t, --test-conf : test configuration for syntax errors and exit
|
55
62
|
-d, --daemonize : run as a daemon
|
56
63
|
-D, --describe-stats : print stats description and exit
|
57
|
-
-v, --
|
64
|
+
-v, --verbose=N : set logging level (default: 5, min: 0, max: 11)
|
58
65
|
-o, --output=S : set logging file (default: stderr)
|
59
66
|
-c, --conf-file=S : set configuration file (default: conf/nutcracker.yml)
|
60
67
|
-s, --stats-port=N : set stats monitoring port (default: 22222)
|
@@ -65,15 +72,15 @@ To build nutcracker from source with _debug logs enabled_ and _assertions disabl
|
|
65
72
|
|
66
73
|
## Zero Copy
|
67
74
|
|
68
|
-
In
|
75
|
+
In twemproxy, all the memory for incoming requests and outgoing responses is allocated in mbuf. Mbuf enables zero-copy because the same buffer on which a request was received from the client is used for forwarding it to the server. Similarly the same mbuf on which a response was received from the server is used for forwarding it to the client.
|
69
76
|
|
70
|
-
Furthermore, memory for mbufs is managed using a reuse pool. This means that once mbuf is allocated, it is not deallocated, but just put back into the reuse pool. By default each mbuf chunk is set to 16K bytes in size. There is a trade-off between the mbuf size and number of concurrent connections
|
77
|
+
Furthermore, memory for mbufs is managed using a reuse pool. This means that once mbuf is allocated, it is not deallocated, but just put back into the reuse pool. By default each mbuf chunk is set to 16K bytes in size. There is a trade-off between the mbuf size and number of concurrent connections twemproxy can support. A large mbuf size reduces the number of read syscalls made by twemproxy when reading requests or responses. However, with a large mbuf size, every active connection would use up 16K bytes of buffer which might be an issue when twemproxy is handling large number of concurrent connections from clients. When twemproxy is meant to handle a large number of concurrent client connections, you should set chunk size to a small value like 512 bytes using the -m or --mbuf-size=N argument.
|
71
78
|
|
72
79
|
## Configuration
|
73
80
|
|
74
|
-
|
81
|
+
Twemproxy can be configured through a YAML file specified by the -c or --conf-file command-line argument on process start. The configuration file is used to specify the server pools and the servers within each pool that twemproxy manages. The configuration files parses and understands the following keys:
|
75
82
|
|
76
|
-
+ **listen**: The listening address and port (name:port or ip:port) for this server pool.
|
83
|
+
+ **listen**: The listening address and port (name:port or ip:port) or an absolute path to sock file (e.g. /var/run/nutcracker.sock) for this server pool.
|
77
84
|
+ **hash**: The name of the hash function. Possible values are:
|
78
85
|
+ one_at_a_time
|
79
86
|
+ md5
|
@@ -87,15 +94,17 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
|
|
87
94
|
+ hsieh
|
88
95
|
+ murmur
|
89
96
|
+ jenkins
|
90
|
-
+ **hash_tag**: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". [Hash tag](notes/recommendation.md#hash-tags)
|
97
|
+
+ **hash_tag**: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". [Hash tag](notes/recommendation.md#hash-tags) enable mapping different keys to the same server as long as the part of the key within the tag is the same.
|
91
98
|
+ **distribution**: The key distribution mode. Possible values are:
|
92
99
|
+ ketama
|
93
100
|
+ modula
|
94
101
|
+ random
|
95
102
|
+ **timeout**: The timeout value in msec that we wait for to establish a connection to the server or receive a response from a server. By default, we wait indefinitely.
|
96
103
|
+ **backlog**: The TCP backlog argument. Defaults to 512.
|
97
|
-
+ **preconnect**: A boolean value that controls if
|
104
|
+
+ **preconnect**: A boolean value that controls if twemproxy should preconnect to all the servers in this pool on process start. Defaults to false.
|
98
105
|
+ **redis**: A boolean value that controls if a server pool speaks redis or memcached protocol. Defaults to false.
|
106
|
+
+ **redis_auth**: Authenticate to the Redis server on connect.
|
107
|
+
+ **redis_db**: The DB number to use on the pool servers. Defaults to 0. Note: Twemproxy will always present itself to clients as DB 0.
|
99
108
|
+ **server_connections**: The maximum number of connections that can be opened to each server. By default, we open at most 1 server connection.
|
100
109
|
+ **auto_eject_hosts**: A boolean value that controls if server should be ejected temporarily when it fails consecutively server_failure_limit times. See [liveness recommendations](notes/recommendation.md#liveness) for information. Defaults to false.
|
101
110
|
+ **server_retry_timeout**: The timeout value in msec to wait for before retrying on a temporarily ejected server, when auto_eject_host is set to true. Defaults to 30000 msec.
|
@@ -103,7 +112,7 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
|
|
103
112
|
+ **servers**: A list of server address, port and weight (name:port:weight or ip:port:weight) for this server pool.
|
104
113
|
|
105
114
|
|
106
|
-
For example, the configuration file in [conf/nutcracker.yml](conf/nutcracker.yml), also shown below, configures 5 server pools with names - _alpha_, _beta_, _gamma_, _delta_ and omega. Clients that intend to send requests to one of the 10 servers in pool delta connect to port 22124 on 127.0.0.1. Clients that intend to send request to one of 2 servers in pool omega connect to unix path /tmp/gamma. Requests sent to pool alpha and omega have no timeout and might require timeout functionality to be implemented on the client side. On the other hand, requests sent to pool beta, gamma and delta timeout after 400 msec, 400 msec and 100 msec respectively when no response is received from the server. Of the 5 server pools, only pools alpha, gamma and delta are configured to use server ejection and hence are resilient to server failures. All the 5 server pools use ketama consistent hashing for key distribution with the key hasher for pools alpha, beta, gamma and delta set to fnv1a_64 while that for pool omega set to hsieh. Also only pool beta uses [nodes names](notes/recommendation.md#node-names-for-consistent-hashing) for consistent hashing, while pool alpha, gamma, delta and omega use 'host:port:weight' for consistent hashing. Finally, only pool alpha and beta can speak redis protocol, while pool gamma, deta and omega speak memcached protocol.
|
115
|
+
For example, the configuration file in [conf/nutcracker.yml](conf/nutcracker.yml), also shown below, configures 5 server pools with names - _alpha_, _beta_, _gamma_, _delta_ and omega. Clients that intend to send requests to one of the 10 servers in pool delta connect to port 22124 on 127.0.0.1. Clients that intend to send request to one of 2 servers in pool omega connect to unix path /tmp/gamma. Requests sent to pool alpha and omega have no timeout and might require timeout functionality to be implemented on the client side. On the other hand, requests sent to pool beta, gamma and delta timeout after 400 msec, 400 msec and 100 msec respectively when no response is received from the server. Of the 5 server pools, only pools alpha, gamma and delta are configured to use server ejection and hence are resilient to server failures. All the 5 server pools use ketama consistent hashing for key distribution with the key hasher for pools alpha, beta, gamma and delta set to fnv1a_64 while that for pool omega set to hsieh. Also only pool beta uses [nodes names](notes/recommendation.md#node-names-for-consistent-hashing) for consistent hashing, while pool alpha, gamma, delta and omega use 'host:port:weight' for consistent hashing. Finally, only pool alpha and beta can speak the redis protocol, while pool gamma, deta and omega speak memcached protocol.
|
107
116
|
|
108
117
|
alpha:
|
109
118
|
listen: 127.0.0.1:22121
|
@@ -173,13 +182,13 @@ For example, the configuration file in [conf/nutcracker.yml](conf/nutcracker.yml
|
|
173
182
|
- 127.0.0.1:11214:100000
|
174
183
|
- 127.0.0.1:11215:1
|
175
184
|
|
176
|
-
Finally, to make writing syntactically correct configuration file easier,
|
185
|
+
Finally, to make writing a syntactically correct configuration file easier, twemproxy provides a command-line argument -t or --test-conf that can be used to test the YAML configuration file for any syntax error.
|
177
186
|
|
178
187
|
## Observability
|
179
188
|
|
180
|
-
Observability in
|
189
|
+
Observability in twemproxy is through logs and stats.
|
181
190
|
|
182
|
-
|
191
|
+
Twemproxy exposes stats at the granularity of server pool and servers per pool through the stats monitoring port. The stats are essentially JSON formatted key-value pairs, with the keys corresponding to counter names. By default stats are exposed on port 22222 and aggregated every 30 seconds. Both these values can be configured on program start using the -c or --conf-file and -i or --stats-interval command-line arguments respectively. You can print the description of all stats exported by using the -D or --describe-stats command-line argument.
|
183
192
|
|
184
193
|
$ nutcracker --describe-stats
|
185
194
|
|
@@ -205,20 +214,31 @@ Nutcracker exposes stats at the granularity of server pool and servers per pool
|
|
205
214
|
out_queue "# requests in outgoing queue"
|
206
215
|
out_queue_bytes "current request bytes in outgoing queue"
|
207
216
|
|
208
|
-
Logging in
|
217
|
+
Logging in twemproxy is only available when twemproxy is built with logging enabled. By default logs are written to stderr. Twemproxy can also be configured to write logs to a specific file through the -o or --output command-line argument. On a running twemproxy, we can turn log levels up and down by sending it SIGTTIN and SIGTTOU signals respectively and reopen log files by sending it SIGHUP signal.
|
209
218
|
|
210
219
|
## Pipelining
|
211
220
|
|
221
|
+
Twemproxy enables proxying multiple client connections onto one or few server connections. This architectural setup makes it ideal for pipelining requests and responses and hence saving on the round trip time.
|
212
222
|
|
213
|
-
|
214
|
-
|
215
|
-
For example, if nutcracker is proxying three client connections onto a single server and we get requests - 'get key\r\n', 'set key 0 0 3\r\nval\r\n' and 'delete key\r\n' on these three connections respectively, nutcracker would try to batch these requests and send them as a single message onto the server connection as 'get key\r\nset key 0 0 3\r\nval\r\ndelete key\r\n'.
|
223
|
+
For example, if twemproxy is proxying three client connections onto a single server and we get requests - 'get key\r\n', 'set key 0 0 3\r\nval\r\n' and 'delete key\r\n' on these three connections respectively, twemproxy would try to batch these requests and send them as a single message onto the server connection as 'get key\r\nset key 0 0 3\r\nval\r\ndelete key\r\n'.
|
216
224
|
|
217
|
-
Pipelining is the reason why
|
225
|
+
Pipelining is the reason why twemproxy ends up doing better in terms of throughput even though it introduces an extra hop between the client and server.
|
218
226
|
|
219
227
|
## Deployment
|
220
228
|
|
221
|
-
If you are deploying
|
229
|
+
If you are deploying twemproxy in production, you might consider reading through the [recommendation document](notes/recommendation.md) to understand the parameters you could tune in twemproxy to run it efficiently in the production environment.
|
230
|
+
|
231
|
+
## Packages
|
232
|
+
|
233
|
+
### Ubuntu
|
234
|
+
|
235
|
+
#### PPA Stable
|
236
|
+
|
237
|
+
https://launchpad.net/~twemproxy/+archive/ubuntu/stable
|
238
|
+
|
239
|
+
#### PPA Daily
|
240
|
+
|
241
|
+
https://launchpad.net/~twemproxy/+archive/ubuntu/daily
|
222
242
|
|
223
243
|
## Utils
|
224
244
|
+ [nagios checks](https://github.com/wanelo/nagios-checks/blob/master/check_twemproxy)
|
@@ -231,8 +251,11 @@ If you are deploying nutcracker in production, you might consider reading throug
|
|
231
251
|
+ [sensu-metrics](https://github.com/sensu/sensu-community-plugins/blob/master/plugins/twemproxy/twemproxy-metrics.rb)
|
232
252
|
+ [redis-mgr](https://github.com/idning/redis-mgr)
|
233
253
|
+ [smitty for twemproxy failover](https://github.com/areina/smitty)
|
254
|
+
+ [Beholder, a Python agent for twemproxy failover](https://github.com/Serekh/beholder)
|
255
|
+
+ [chef cookbook](https://supermarket.getchef.com/cookbooks/twemproxy)
|
256
|
+
+ [twemsentinel] (https://github.com/yak0/twemsentinel)
|
234
257
|
|
235
|
-
##
|
258
|
+
## Companies using Twemproxy in Production
|
236
259
|
+ [Pinterest](http://pinterest.com/)
|
237
260
|
+ [Tumblr](https://www.tumblr.com/)
|
238
261
|
+ [Twitter](https://twitter.com/)
|
@@ -250,6 +273,18 @@ If you are deploying nutcracker in production, you might consider reading throug
|
|
250
273
|
+ [3scale.net](http://3scale.net)
|
251
274
|
+ [Ooyala](http://www.ooyala.com)
|
252
275
|
+ [Twitch](http://twitch.tv)
|
276
|
+
+ [Socrata](http://www.socrata.com/)
|
277
|
+
+ [Hootsuite](http://hootsuite.com/)
|
278
|
+
+ [Trivago](http://www.trivago.com/)
|
279
|
+
+ [Machinezone](http://www.machinezone.com)
|
280
|
+
+ [Flickr](https://www.flickr.com)
|
281
|
+
+ [Yahoo!](https://www.yahoo.com)
|
282
|
+
+ [Path](https://path.com)
|
283
|
+
+ [AOL](http://engineering.aol.com/)
|
284
|
+
+ [Soysuper](https://soysuper.com/)
|
285
|
+
+ [Vinted](http://vinted.com/)
|
286
|
+
+ [Poshmark](https://poshmark.com/)
|
287
|
+
+ [FanDuel](https://www.fanduel.com/)
|
253
288
|
|
254
289
|
## Issues and Support
|
255
290
|
|