nutcracker 0.4.0.16 → 0.4.1.18

Sign up to get free protection for your applications and to get access to all the features.
Files changed (121) hide show
  1. checksums.yaml +4 -4
  2. data/Rakefile +3 -3
  3. data/ext/nutcracker/ChangeLog +33 -8
  4. data/ext/nutcracker/Makefile.in +34 -21
  5. data/ext/nutcracker/README.md +61 -26
  6. data/ext/nutcracker/aclocal.m4 +34 -31
  7. data/ext/nutcracker/autom4te.cache/output.0 +1875 -1330
  8. data/ext/nutcracker/autom4te.cache/output.1 +1875 -1330
  9. data/ext/nutcracker/autom4te.cache/requests +232 -451
  10. data/ext/nutcracker/autom4te.cache/traces.0 +2256 -2129
  11. data/ext/nutcracker/autom4te.cache/traces.1 +73 -59
  12. data/ext/nutcracker/config.h.in +1 -2
  13. data/ext/nutcracker/config.h.in~ +333 -0
  14. data/ext/nutcracker/config/compile +1 -1
  15. data/ext/nutcracker/config/config.guess +13 -160
  16. data/ext/nutcracker/config/config.sub +25 -11
  17. data/ext/nutcracker/config/depcomp +1 -1
  18. data/ext/nutcracker/config/install-sh +170 -196
  19. data/ext/nutcracker/config/ltmain.sh +3509 -2018
  20. data/ext/nutcracker/config/missing +1 -1
  21. data/ext/nutcracker/configure +1874 -1329
  22. data/ext/nutcracker/configure.ac +3 -2
  23. data/ext/nutcracker/contrib/Makefile.in +18 -5
  24. data/ext/nutcracker/contrib/yaml-0.1.4/LICENSE +19 -0
  25. data/ext/nutcracker/contrib/yaml-0.1.4/Makefile.am +20 -0
  26. data/ext/nutcracker/contrib/yaml-0.1.4/Makefile.in +832 -0
  27. data/ext/nutcracker/contrib/yaml-0.1.4/README +27 -0
  28. data/ext/nutcracker/contrib/yaml-0.1.4/aclocal.m4 +1157 -0
  29. data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/output.0 +13342 -0
  30. data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/output.1 +14611 -0
  31. data/ext/nutcracker/{autom4te.cache → contrib/yaml-0.1.4/autom4te.cache}/output.2 +3465 -8761
  32. data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/requests +516 -0
  33. data/ext/nutcracker/{autom4te.cache/traces.2 → contrib/yaml-0.1.4/autom4te.cache/traces.0} +662 -698
  34. data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/traces.1 +577 -0
  35. data/ext/nutcracker/contrib/yaml-0.1.4/autom4te.cache/traces.2 +2721 -0
  36. data/ext/nutcracker/contrib/yaml-0.1.4/config.h.in +79 -0
  37. data/ext/nutcracker/contrib/yaml-0.1.4/config.h.in~ +80 -0
  38. data/ext/nutcracker/contrib/yaml-0.1.4/config/compile +347 -0
  39. data/ext/nutcracker/contrib/yaml-0.1.4/config/config.guess +1421 -0
  40. data/ext/nutcracker/contrib/yaml-0.1.4/config/config.sub +1807 -0
  41. data/ext/nutcracker/contrib/yaml-0.1.4/config/depcomp +791 -0
  42. data/ext/nutcracker/contrib/yaml-0.1.4/config/install-sh +501 -0
  43. data/ext/nutcracker/contrib/yaml-0.1.4/config/ltmain.sh +11147 -0
  44. data/ext/nutcracker/contrib/yaml-0.1.4/config/missing +215 -0
  45. data/ext/nutcracker/contrib/yaml-0.1.4/config/test-driver +148 -0
  46. data/ext/nutcracker/contrib/yaml-0.1.4/configure +14611 -0
  47. data/ext/nutcracker/contrib/yaml-0.1.4/configure.ac +75 -0
  48. data/ext/nutcracker/contrib/yaml-0.1.4/doc/doxygen.cfg +222 -0
  49. data/ext/nutcracker/contrib/yaml-0.1.4/include/yaml.h +1971 -0
  50. data/ext/nutcracker/contrib/yaml-0.1.4/m4/libtool.m4 +8369 -0
  51. data/ext/nutcracker/contrib/yaml-0.1.4/m4/ltoptions.m4 +437 -0
  52. data/ext/nutcracker/contrib/yaml-0.1.4/m4/ltsugar.m4 +124 -0
  53. data/ext/nutcracker/contrib/yaml-0.1.4/m4/ltversion.m4 +23 -0
  54. data/ext/nutcracker/contrib/yaml-0.1.4/m4/lt~obsolete.m4 +99 -0
  55. data/ext/nutcracker/contrib/yaml-0.1.4/src/Makefile.am +4 -0
  56. data/ext/nutcracker/contrib/yaml-0.1.4/src/Makefile.in +600 -0
  57. data/ext/nutcracker/contrib/yaml-0.1.4/src/api.c +1392 -0
  58. data/ext/nutcracker/contrib/yaml-0.1.4/src/dumper.c +394 -0
  59. data/ext/nutcracker/contrib/yaml-0.1.4/src/emitter.c +2329 -0
  60. data/ext/nutcracker/contrib/yaml-0.1.4/src/loader.c +432 -0
  61. data/ext/nutcracker/contrib/yaml-0.1.4/src/parser.c +1374 -0
  62. data/ext/nutcracker/contrib/yaml-0.1.4/src/reader.c +465 -0
  63. data/ext/nutcracker/contrib/yaml-0.1.4/src/scanner.c +3570 -0
  64. data/ext/nutcracker/contrib/yaml-0.1.4/src/writer.c +141 -0
  65. data/ext/nutcracker/contrib/yaml-0.1.4/src/yaml_private.h +640 -0
  66. data/ext/nutcracker/contrib/yaml-0.1.4/tests/Makefile.am +8 -0
  67. data/ext/nutcracker/contrib/yaml-0.1.4/tests/Makefile.in +1083 -0
  68. data/ext/nutcracker/contrib/yaml-0.1.4/tests/example-deconstructor-alt.c +800 -0
  69. data/ext/nutcracker/contrib/yaml-0.1.4/tests/example-deconstructor.c +1130 -0
  70. data/ext/nutcracker/contrib/yaml-0.1.4/tests/example-reformatter-alt.c +217 -0
  71. data/ext/nutcracker/contrib/yaml-0.1.4/tests/example-reformatter.c +202 -0
  72. data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-dumper.c +311 -0
  73. data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-emitter.c +327 -0
  74. data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-loader.c +63 -0
  75. data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-parser.c +63 -0
  76. data/ext/nutcracker/contrib/yaml-0.1.4/tests/run-scanner.c +63 -0
  77. data/ext/nutcracker/contrib/yaml-0.1.4/tests/test-reader.c +354 -0
  78. data/ext/nutcracker/contrib/yaml-0.1.4/tests/test-version.c +29 -0
  79. data/ext/nutcracker/m4/libtool.m4 +1474 -1087
  80. data/ext/nutcracker/m4/ltoptions.m4 +90 -37
  81. data/ext/nutcracker/m4/ltsugar.m4 +4 -3
  82. data/ext/nutcracker/m4/ltversion.m4 +6 -6
  83. data/ext/nutcracker/m4/lt~obsolete.m4 +4 -3
  84. data/ext/nutcracker/man/nutcracker.8 +1 -1
  85. data/ext/nutcracker/notes/memcache.md +162 -0
  86. data/ext/nutcracker/notes/recommendation.md +10 -5
  87. data/ext/nutcracker/notes/redis.md +23 -9
  88. data/ext/nutcracker/scripts/nutcracker.init +10 -0
  89. data/ext/nutcracker/scripts/nutcracker.init.debian +83 -0
  90. data/ext/nutcracker/scripts/nutcracker.spec +36 -2
  91. data/ext/nutcracker/scripts/redis-check.sh +4 -0
  92. data/ext/nutcracker/src/Makefile.am +6 -1
  93. data/ext/nutcracker/src/Makefile.in +26 -12
  94. data/ext/nutcracker/src/event/Makefile.in +19 -6
  95. data/ext/nutcracker/src/hashkit/Makefile.in +19 -6
  96. data/ext/nutcracker/src/hashkit/nc_jenkins.c +1 -1
  97. data/ext/nutcracker/src/hashkit/nc_ketama.c +3 -3
  98. data/ext/nutcracker/src/nc.c +1 -1
  99. data/ext/nutcracker/src/nc_conf.c +67 -19
  100. data/ext/nutcracker/src/nc_conf.h +9 -4
  101. data/ext/nutcracker/src/nc_connection.c +35 -2
  102. data/ext/nutcracker/src/nc_connection.h +53 -47
  103. data/ext/nutcracker/src/nc_core.c +8 -1
  104. data/ext/nutcracker/src/nc_message.c +23 -7
  105. data/ext/nutcracker/src/nc_message.h +24 -1
  106. data/ext/nutcracker/src/nc_proxy.c +14 -3
  107. data/ext/nutcracker/src/nc_rbtree.c +1 -5
  108. data/ext/nutcracker/src/nc_request.c +58 -10
  109. data/ext/nutcracker/src/nc_response.c +27 -4
  110. data/ext/nutcracker/src/nc_server.c +33 -5
  111. data/ext/nutcracker/src/nc_server.h +10 -9
  112. data/ext/nutcracker/src/nc_string.h +17 -0
  113. data/ext/nutcracker/src/nc_util.c +5 -1
  114. data/ext/nutcracker/src/proto/Makefile.in +19 -6
  115. data/ext/nutcracker/src/proto/nc_memcache.c +76 -12
  116. data/ext/nutcracker/src/proto/nc_proto.h +9 -0
  117. data/ext/nutcracker/src/proto/nc_redis.c +400 -18
  118. data/lib/nutcracker.rb +1 -1
  119. data/lib/nutcracker/version.rb +1 -1
  120. metadata +61 -6
  121. data/ext/nutcracker/notes/memcache.txt +0 -123
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 0131d3faa17fa5c4bae74bf0928e83b67526c88c
4
- data.tar.gz: fff7fa05ce68995614938c5119c6e5e7ebdebf80
3
+ metadata.gz: 96ee5eec694e7984421bb7cc24f4cb081ffebea3
4
+ data.tar.gz: 357729667f8ea82d7c6b39f14ada570040f675c9
5
5
  SHA512:
6
- metadata.gz: 30fd3866cdba768abd2671cf3de3f7e1942803879915c1e05f19a3dfe5d72ee355f7b42adec59439310038d36cba1d807a14149c06f69cefd300aed389df6c16
7
- data.tar.gz: e6f4511267e566fa3c8ab105bc4523433bd367d3b55126698d9649094f5b5fac288450346db8f8b6a130f97604de62dee04de47e2bce6f8ecfa62910d0663723
6
+ metadata.gz: 22cd6aea9938befd72afb512e5b0d003795d29760618c39c7d76b1e433f3dcafec84a59063dd5d1b820cb36758ded7a335e514ea4202a072714bd6a7a168ae7c
7
+ data.tar.gz: a99b46cb63b4b447a5ba588317fc0bf7bc411a976ae7f72542b304b521a5b108bdb63f8f5a6fb4f644a6ff54cc47debbd847c70b37b12b0115323a69c3282253
data/Rakefile CHANGED
@@ -10,12 +10,12 @@ sversion = Nutcracker.version.split(".")[0..2].join(".")
10
10
 
11
11
  desc "Download Nutcracker c app"
12
12
  task :download do
13
- "twemproxy-#{sversion}.tar.gz".tap do |tarball|
13
+ "nutcracker-#{sversion}.tar.gz".tap do |tarball|
14
14
  sh "mkdir ext" unless File.directory? "ext"
15
15
  sh "rm -rf ext/nutcracker"
16
- sh "wget https://github.com/twitter/twemproxy/archive/v#{sversion}.tar.gz -O #{tarball}"
16
+ #sh "wget https://github.com/twitter/twemproxy/archive/v#{sversion}.tar.gz -O #{tarball}"
17
17
  sh "tar -zxvf #{tarball}"
18
- sh "mv twemproxy-#{sversion} ext/nutcracker"
18
+ sh "mv nutcracker-#{sversion} ext/nutcracker"
19
19
  Dir.chdir("ext/nutcracker") do
20
20
  sh "autoreconf -fvi"
21
21
  c = File.read("configure").gsub("-${am__api_version}","")
@@ -1,3 +1,36 @@
1
+ 2015-22-06 Manju Rajashekhar <manj@cs.stanford.edu>
2
+ * twemproxy: version 0.4.1 release
3
+ redis_auth is only valid for a redis pool
4
+ getaddrinfo returns non-zero +ve value on error
5
+ fix-hang-when-command-only (charsyam)
6
+ fix bug crash when get command without key and whitespace (charsyam)
7
+ mark server as failed on protocol level transiet failures like -OOM, -LOADING, etc
8
+ implemented support for parsing fine grained redis error response
9
+ remove redundant conditional judgement in rbtree deletion (leo ma)
10
+ fix bug mset has invalid pair (charsyam)
11
+ fix bug mset has invalid pair (charsyam)
12
+ temp fix a core on kqueue (idning)
13
+ support "touch" command for memcached (panmiaocai)
14
+ fix redis parse rsp bug (charsyam)
15
+ SORT command can take multiple arguments. So it should be part of redis_argn() and not redis_arg0()
16
+ remove incorrect assert because client could send data after sending a quit request which must be discarded
17
+ allow file permissions to be set for UNIX domain listening socket (ori liveneh)
18
+ return error if formatted is greater than mbuf size by using nc_vsnprintf() in msg_prepend_format()
19
+ fix req_make_reply on msg_get, mark it as response (idning)
20
+ redis database select upon connect (arne claus)
21
+ redis_auth (charsyam)
22
+ allow null key(empty key) (idning)
23
+ fix core on invalid mset like "mset a a a" (idning)
24
+
25
+ 2014-18-10 idning <idning@gmail.com>
26
+ * twemproxy: version 0.4.0 release
27
+ mget improve (idning)
28
+ many new commands supported: LEX, PFADD, PFMERGE, SORT, PING, QUIT, SCAN... (mattrobenolt, areina, idning)
29
+ handle max open file limit(allenlz)
30
+ add notice-log and use ms time in log(idning)
31
+ fix bug in string_compare (andyqzb)
32
+ fix deadlock in sighandler (idning)
33
+
1
34
  2013-20-12 Manju Rajashekhar <manj@cs.stanford.edu>
2
35
  * twemproxy: version 0.3.0 release
3
36
  SRANDMEMBER support for the optional count argument (mkhq)
@@ -28,12 +61,10 @@
28
61
  add support for hash tags
29
62
 
30
63
  2012-18-10 Manju Rajashekhar <manj@twitter.com>
31
-
32
64
  * twemproxy: version 0.2.2 release
33
65
  fix the off-by-one error when calculating redis key length
34
66
 
35
67
  2012-12-10 Manju Rajashekhar <manj@twitter.com>
36
-
37
68
  * twemproxy: version 0.2.1 release
38
69
  don't use buf in conf_add_server
39
70
  allow an optional instance name for consistent hashing (charsyam)
@@ -41,7 +72,6 @@
41
72
  add stats-bind-any -a option (charsyam)
42
73
 
43
74
  2012-12-03 Manju Rajashekhar <manj@twitter.com>
44
-
45
75
  * twemproxy: version 0.2.0 release
46
76
  add -D or --describe-stats command-line argument to print stats description
47
77
  redis support in twemproxy
@@ -54,7 +84,6 @@
54
84
  delete stats tracking memcache requests and responses; stats module no longer tracks protocol related stats
55
85
 
56
86
  2012-10-27 Manju Rajashekhar <manj@twitter.com>
57
-
58
87
  * twemproxy: version 0.1.20 release
59
88
  on msg_repair, msg->pos should point to nbuf->pos and not nbuf->last
60
89
  refactor memcache parsing code into proto directory
@@ -63,23 +92,19 @@
63
92
  fix log_hexdump and loga_hexdump
64
93
 
65
94
  2012-07-31 Manju Rajashekhar <manj@twitter.com>
66
-
67
95
  * twemproxy: version 0.1.19 release
68
96
  close server connection on a stray response (yashh, bmatheny)
69
97
 
70
98
  2012-06-19 Manju Rajashekhar <manj@twitter.com>
71
-
72
99
  * twemproxy: version 0.1.18 release
73
100
  command line option to set mbuf chunk size
74
101
 
75
102
  2012-05-09 Manju Rajashekhar <manj@twitter.com>
76
-
77
103
  * twemproxy: version 0.1.17 release
78
104
  use _exit(0) instead of exit(0) when daemonizing
79
105
  use loga instead of log_stderr in nc_stacktrace
80
106
 
81
107
  2012-02-09 Manju Rajashekhar <manj@twitter.com>
82
-
83
108
  * twemproxy: version 0.1.16 release
84
109
  twemproxy (aka nutcracker) is a fast and lightweight proxy for memcached protocol.
85
110
 
@@ -1,7 +1,7 @@
1
- # Makefile.in generated by automake 1.14.1 from Makefile.am.
1
+ # Makefile.in generated by automake 1.15 from Makefile.am.
2
2
  # @configure_input@
3
3
 
4
- # Copyright (C) 1994-2013 Free Software Foundation, Inc.
4
+ # Copyright (C) 1994-2014 Free Software Foundation, Inc.
5
5
 
6
6
  # This Makefile.in is free software; the Free Software Foundation
7
7
  # gives unlimited permission to copy and/or distribute it,
@@ -14,7 +14,17 @@
14
14
 
15
15
  @SET_MAKE@
16
16
  VPATH = @srcdir@
17
- am__is_gnu_make = test -n '$(MAKEFILE_LIST)' && test -n '$(MAKELEVEL)'
17
+ am__is_gnu_make = { \
18
+ if test -z '$(MAKELEVEL)'; then \
19
+ false; \
20
+ elif test -n '$(MAKE_HOST)'; then \
21
+ true; \
22
+ elif test -n '$(MAKE_VERSION)' && test -n '$(CURDIR)'; then \
23
+ true; \
24
+ else \
25
+ false; \
26
+ fi; \
27
+ }
18
28
  am__make_running_with_option = \
19
29
  case $${target_option-} in \
20
30
  ?) ;; \
@@ -78,15 +88,6 @@ POST_UNINSTALL = :
78
88
  build_triplet = @build@
79
89
  host_triplet = @host@
80
90
  subdir = .
81
- DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am \
82
- $(top_srcdir)/configure $(am__configure_deps) \
83
- $(srcdir)/config.h.in $(dist_man_MANS) ChangeLog \
84
- config/compile config/config.guess config/config.sub \
85
- config/install-sh config/missing config/ltmain.sh \
86
- $(top_srcdir)/config/compile $(top_srcdir)/config/config.guess \
87
- $(top_srcdir)/config/config.sub \
88
- $(top_srcdir)/config/install-sh $(top_srcdir)/config/ltmain.sh \
89
- $(top_srcdir)/config/missing
90
91
  ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
91
92
  am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \
92
93
  $(top_srcdir)/m4/ltoptions.m4 $(top_srcdir)/m4/ltsugar.m4 \
@@ -94,6 +95,8 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \
94
95
  $(top_srcdir)/configure.ac
95
96
  am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
96
97
  $(ACLOCAL_M4)
98
+ DIST_COMMON = $(srcdir)/Makefile.am $(top_srcdir)/configure \
99
+ $(am__configure_deps) $(am__DIST_COMMON)
97
100
  am__CONFIG_DISTCLEAN_FILES = config.status config.cache config.log \
98
101
  configure.lineno config.status.lineno
99
102
  mkinstalldirs = $(install_sh) -d
@@ -188,6 +191,14 @@ ETAGS = etags
188
191
  CTAGS = ctags
189
192
  CSCOPE = cscope
190
193
  DIST_SUBDIRS = $(SUBDIRS)
194
+ am__DIST_COMMON = $(dist_man_MANS) $(srcdir)/Makefile.in \
195
+ $(srcdir)/config.h.in $(top_srcdir)/config/compile \
196
+ $(top_srcdir)/config/config.guess \
197
+ $(top_srcdir)/config/config.sub \
198
+ $(top_srcdir)/config/install-sh $(top_srcdir)/config/ltmain.sh \
199
+ $(top_srcdir)/config/missing ChangeLog config/compile \
200
+ config/config.guess config/config.sub config/install-sh \
201
+ config/ltmain.sh config/missing
191
202
  DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST)
192
203
  distdir = $(PACKAGE)-$(VERSION)
193
204
  top_distdir = $(distdir)
@@ -273,6 +284,7 @@ LIBTOOL = @LIBTOOL@
273
284
  LIPO = @LIPO@
274
285
  LN_S = @LN_S@
275
286
  LTLIBOBJS = @LTLIBOBJS@
287
+ LT_SYS_LIBRARY_PATH = @LT_SYS_LIBRARY_PATH@
276
288
  MAKEINFO = @MAKEINFO@
277
289
  MANIFEST_TOOL = @MANIFEST_TOOL@
278
290
  MKDIR_P = @MKDIR_P@
@@ -374,7 +386,6 @@ $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps)
374
386
  echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign Makefile'; \
375
387
  $(am__cd) $(top_srcdir) && \
376
388
  $(AUTOMAKE) --foreign Makefile
377
- .PRECIOUS: Makefile
378
389
  Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status
379
390
  @case '$?' in \
380
391
  *config.status*) \
@@ -648,15 +659,15 @@ dist-xz: distdir
648
659
  $(am__post_remove_distdir)
649
660
 
650
661
  dist-tarZ: distdir
651
- @echo WARNING: "Support for shar distribution archives is" \
652
- "deprecated." >&2
662
+ @echo WARNING: "Support for distribution archives compressed with" \
663
+ "legacy program 'compress' is deprecated." >&2
653
664
  @echo WARNING: "It will be removed altogether in Automake 2.0" >&2
654
665
  tardir=$(distdir) && $(am__tar) | compress -c >$(distdir).tar.Z
655
666
  $(am__post_remove_distdir)
656
667
 
657
668
  dist-shar: distdir
658
- @echo WARNING: "Support for distribution archives compressed with" \
659
- "legacy program 'compress' is deprecated." >&2
669
+ @echo WARNING: "Support for shar distribution archives is" \
670
+ "deprecated." >&2
660
671
  @echo WARNING: "It will be removed altogether in Automake 2.0" >&2
661
672
  shar $(distdir) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).shar.gz
662
673
  $(am__post_remove_distdir)
@@ -692,17 +703,17 @@ distcheck: dist
692
703
  esac
693
704
  chmod -R a-w $(distdir)
694
705
  chmod u+w $(distdir)
695
- mkdir $(distdir)/_build $(distdir)/_inst
706
+ mkdir $(distdir)/_build $(distdir)/_build/sub $(distdir)/_inst
696
707
  chmod a-w $(distdir)
697
708
  test -d $(distdir)/_build || exit 0; \
698
709
  dc_install_base=`$(am__cd) $(distdir)/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \
699
710
  && dc_destdir="$${TMPDIR-/tmp}/am-dc-$$$$/" \
700
711
  && am__cwd=`pwd` \
701
- && $(am__cd) $(distdir)/_build \
702
- && ../configure \
712
+ && $(am__cd) $(distdir)/_build/sub \
713
+ && ../../configure \
703
714
  $(AM_DISTCHECK_CONFIGURE_FLAGS) \
704
715
  $(DISTCHECK_CONFIGURE_FLAGS) \
705
- --srcdir=.. --prefix="$$dc_install_base" \
716
+ --srcdir=../.. --prefix="$$dc_install_base" \
706
717
  && $(MAKE) $(AM_MAKEFLAGS) \
707
718
  && $(MAKE) $(AM_MAKEFLAGS) dvi \
708
719
  && $(MAKE) $(AM_MAKEFLAGS) check \
@@ -885,6 +896,8 @@ uninstall-man: uninstall-man8
885
896
  mostlyclean-libtool pdf pdf-am ps ps-am tags tags-am uninstall \
886
897
  uninstall-am uninstall-man uninstall-man8
887
898
 
899
+ .PRECIOUS: Makefile
900
+
888
901
 
889
902
  # Tell versions [3.59,3.63) of GNU make to not export all variables.
890
903
  # Otherwise a system limit (for SysV at least) may be exceeded.
@@ -1,30 +1,37 @@
1
1
  # twemproxy (nutcracker) [![Build Status](https://secure.travis-ci.org/twitter/twemproxy.png)](http://travis-ci.org/twitter/twemproxy)
2
2
 
3
- **twemproxy** (pronounced "two-em-proxy"), aka **nutcracker** is a fast and lightweight proxy for [memcached](http://www.memcached.org/) and [redis](http://redis.io/) protocol. It was primarily built to reduce the connection count on the backend caching servers.
3
+ **twemproxy** (pronounced "two-em-proxy"), aka **nutcracker** is a fast and lightweight proxy for [memcached](http://www.memcached.org/) and [redis](http://redis.io/) protocol. It was built primarily to reduce the number of connections to the caching servers on the backend. This, together with protocol pipelining and sharding enables you to horizontally scale your distributed caching architecture.
4
4
 
5
5
  ## Build
6
6
 
7
- To build nutcracker from [distribution tarball](http://code.google.com/p/twemproxy/downloads/list):
7
+ To build twemproxy from [distribution tarball](https://drive.google.com/open?id=0B6pVMMV5F5dfMUdJV25abllhUWM&authuser=0):
8
8
 
9
9
  $ ./configure
10
10
  $ make
11
11
  $ sudo make install
12
12
 
13
- To build nutcracker from [distribution tarball](http://code.google.com/p/twemproxy/downloads/list) in _debug mode_:
13
+ To build twemproxy from [distribution tarball](https://drive.google.com/open?id=0B6pVMMV5F5dfMUdJV25abllhUWM&authuser=0) in _debug mode_:
14
14
 
15
15
  $ CFLAGS="-ggdb3 -O0" ./configure --enable-debug=full
16
16
  $ make
17
17
  $ sudo make install
18
18
 
19
- To build nutcracker from source with _debug logs enabled_ and _assertions disabled_:
19
+ To build twemproxy from source with _debug logs enabled_ and _assertions enabled_:
20
20
 
21
21
  $ git clone git@github.com:twitter/twemproxy.git
22
22
  $ cd twemproxy
23
23
  $ autoreconf -fvi
24
- $ ./configure --enable-debug=log
24
+ $ ./configure --enable-debug=full
25
25
  $ make
26
26
  $ src/nutcracker -h
27
27
 
28
+ A quick checklist:
29
+
30
+ + Use newer version of gcc (older version of gcc has problems)
31
+ + Use CFLAGS="-O1" ./configure && make
32
+ + Use CFLAGS="-O3 -fno-strict-aliasing" ./configure && make
33
+ + `autoreconf -fvi && ./configure` needs `automake` and `libtool` to be installed
34
+
28
35
  ## Features
29
36
 
30
37
  + Fast.
@@ -35,12 +42,12 @@ To build nutcracker from source with _debug logs enabled_ and _assertions disabl
35
42
  + Supports proxying to multiple servers.
36
43
  + Supports multiple server pools simultaneously.
37
44
  + Shard data automatically across multiple servers.
38
- + Implements the complete [memcached ascii](notes/memcache.txt) and [redis](notes/redis.md) protocol.
45
+ + Implements the complete [memcached ascii](notes/memcache.md) and [redis](notes/redis.md) protocol.
39
46
  + Easy configuration of server pools through a YAML file.
40
47
  + Supports multiple hashing modes including consistent hashing and distribution.
41
48
  + Can be configured to disable nodes on failures.
42
- + Observability through stats exposed on stats monitoring port.
43
- + Works with Linux, *BSD, OS X and Solaris (SmartOS)
49
+ + Observability via stats exposed on the stats monitoring port.
50
+ + Works with Linux, *BSD, OS X and SmartOS (Solaris)
44
51
 
45
52
  ## Help
46
53
 
@@ -54,7 +61,7 @@ To build nutcracker from source with _debug logs enabled_ and _assertions disabl
54
61
  -t, --test-conf : test configuration for syntax errors and exit
55
62
  -d, --daemonize : run as a daemon
56
63
  -D, --describe-stats : print stats description and exit
57
- -v, --verbosity=N : set logging level (default: 5, min: 0, max: 11)
64
+ -v, --verbose=N : set logging level (default: 5, min: 0, max: 11)
58
65
  -o, --output=S : set logging file (default: stderr)
59
66
  -c, --conf-file=S : set configuration file (default: conf/nutcracker.yml)
60
67
  -s, --stats-port=N : set stats monitoring port (default: 22222)
@@ -65,15 +72,15 @@ To build nutcracker from source with _debug logs enabled_ and _assertions disabl
65
72
 
66
73
  ## Zero Copy
67
74
 
68
- In nutcracker, all the memory for incoming requests and outgoing responses is allocated in mbuf. Mbuf enables zero-copy because the same buffer on which a request was received from the client is used for forwarding it to the server. Similarly the same mbuf on which a response was received from the server is used for forwarding it to the client.
75
+ In twemproxy, all the memory for incoming requests and outgoing responses is allocated in mbuf. Mbuf enables zero-copy because the same buffer on which a request was received from the client is used for forwarding it to the server. Similarly the same mbuf on which a response was received from the server is used for forwarding it to the client.
69
76
 
70
- Furthermore, memory for mbufs is managed using a reuse pool. This means that once mbuf is allocated, it is not deallocated, but just put back into the reuse pool. By default each mbuf chunk is set to 16K bytes in size. There is a trade-off between the mbuf size and number of concurrent connections nutcracker can support. A large mbuf size reduces the number of read syscalls made by nutcracker when reading requests or responses. However, with large mbuf size, every active connection would use up 16K bytes of buffer which might be an issue when nutcracker is handling large number of concurrent connections from clients. When nutcracker is meant to handle a large number of concurrent client connections, you should set chunk size to a small value like 512 bytes using the -m or --mbuf-size=N argument.
77
+ Furthermore, memory for mbufs is managed using a reuse pool. This means that once mbuf is allocated, it is not deallocated, but just put back into the reuse pool. By default each mbuf chunk is set to 16K bytes in size. There is a trade-off between the mbuf size and number of concurrent connections twemproxy can support. A large mbuf size reduces the number of read syscalls made by twemproxy when reading requests or responses. However, with a large mbuf size, every active connection would use up 16K bytes of buffer which might be an issue when twemproxy is handling large number of concurrent connections from clients. When twemproxy is meant to handle a large number of concurrent client connections, you should set chunk size to a small value like 512 bytes using the -m or --mbuf-size=N argument.
71
78
 
72
79
  ## Configuration
73
80
 
74
- nutcracker can be configured through a YAML file specified by the -c or --conf-file command-line argument on process start. The configuration file is used to specify the server pools and the servers within each pool that nutcracker manages. The configuration files parses and understands the following keys:
81
+ Twemproxy can be configured through a YAML file specified by the -c or --conf-file command-line argument on process start. The configuration file is used to specify the server pools and the servers within each pool that twemproxy manages. The configuration files parses and understands the following keys:
75
82
 
76
- + **listen**: The listening address and port (name:port or ip:port) for this server pool.
83
+ + **listen**: The listening address and port (name:port or ip:port) or an absolute path to sock file (e.g. /var/run/nutcracker.sock) for this server pool.
77
84
  + **hash**: The name of the hash function. Possible values are:
78
85
  + one_at_a_time
79
86
  + md5
@@ -87,15 +94,17 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
87
94
  + hsieh
88
95
  + murmur
89
96
  + jenkins
90
- + **hash_tag**: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". [Hash tag](notes/recommendation.md#hash-tags) enable mapping different keys to the same server as long as the part of the key within the tag is the same.
97
+ + **hash_tag**: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". [Hash tag](notes/recommendation.md#hash-tags) enable mapping different keys to the same server as long as the part of the key within the tag is the same.
91
98
  + **distribution**: The key distribution mode. Possible values are:
92
99
  + ketama
93
100
  + modula
94
101
  + random
95
102
  + **timeout**: The timeout value in msec that we wait for to establish a connection to the server or receive a response from a server. By default, we wait indefinitely.
96
103
  + **backlog**: The TCP backlog argument. Defaults to 512.
97
- + **preconnect**: A boolean value that controls if nutcracker should preconnect to all the servers in this pool on process start. Defaults to false.
104
+ + **preconnect**: A boolean value that controls if twemproxy should preconnect to all the servers in this pool on process start. Defaults to false.
98
105
  + **redis**: A boolean value that controls if a server pool speaks redis or memcached protocol. Defaults to false.
106
+ + **redis_auth**: Authenticate to the Redis server on connect.
107
+ + **redis_db**: The DB number to use on the pool servers. Defaults to 0. Note: Twemproxy will always present itself to clients as DB 0.
99
108
  + **server_connections**: The maximum number of connections that can be opened to each server. By default, we open at most 1 server connection.
100
109
  + **auto_eject_hosts**: A boolean value that controls if server should be ejected temporarily when it fails consecutively server_failure_limit times. See [liveness recommendations](notes/recommendation.md#liveness) for information. Defaults to false.
101
110
  + **server_retry_timeout**: The timeout value in msec to wait for before retrying on a temporarily ejected server, when auto_eject_host is set to true. Defaults to 30000 msec.
@@ -103,7 +112,7 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
103
112
  + **servers**: A list of server address, port and weight (name:port:weight or ip:port:weight) for this server pool.
104
113
 
105
114
 
106
- For example, the configuration file in [conf/nutcracker.yml](conf/nutcracker.yml), also shown below, configures 5 server pools with names - _alpha_, _beta_, _gamma_, _delta_ and omega. Clients that intend to send requests to one of the 10 servers in pool delta connect to port 22124 on 127.0.0.1. Clients that intend to send request to one of 2 servers in pool omega connect to unix path /tmp/gamma. Requests sent to pool alpha and omega have no timeout and might require timeout functionality to be implemented on the client side. On the other hand, requests sent to pool beta, gamma and delta timeout after 400 msec, 400 msec and 100 msec respectively when no response is received from the server. Of the 5 server pools, only pools alpha, gamma and delta are configured to use server ejection and hence are resilient to server failures. All the 5 server pools use ketama consistent hashing for key distribution with the key hasher for pools alpha, beta, gamma and delta set to fnv1a_64 while that for pool omega set to hsieh. Also only pool beta uses [nodes names](notes/recommendation.md#node-names-for-consistent-hashing) for consistent hashing, while pool alpha, gamma, delta and omega use 'host:port:weight' for consistent hashing. Finally, only pool alpha and beta can speak redis protocol, while pool gamma, deta and omega speak memcached protocol.
115
+ For example, the configuration file in [conf/nutcracker.yml](conf/nutcracker.yml), also shown below, configures 5 server pools with names - _alpha_, _beta_, _gamma_, _delta_ and omega. Clients that intend to send requests to one of the 10 servers in pool delta connect to port 22124 on 127.0.0.1. Clients that intend to send request to one of 2 servers in pool omega connect to unix path /tmp/gamma. Requests sent to pool alpha and omega have no timeout and might require timeout functionality to be implemented on the client side. On the other hand, requests sent to pool beta, gamma and delta timeout after 400 msec, 400 msec and 100 msec respectively when no response is received from the server. Of the 5 server pools, only pools alpha, gamma and delta are configured to use server ejection and hence are resilient to server failures. All the 5 server pools use ketama consistent hashing for key distribution with the key hasher for pools alpha, beta, gamma and delta set to fnv1a_64 while that for pool omega set to hsieh. Also only pool beta uses [nodes names](notes/recommendation.md#node-names-for-consistent-hashing) for consistent hashing, while pool alpha, gamma, delta and omega use 'host:port:weight' for consistent hashing. Finally, only pool alpha and beta can speak the redis protocol, while pool gamma, deta and omega speak memcached protocol.
107
116
 
108
117
  alpha:
109
118
  listen: 127.0.0.1:22121
@@ -173,13 +182,13 @@ For example, the configuration file in [conf/nutcracker.yml](conf/nutcracker.yml
173
182
  - 127.0.0.1:11214:100000
174
183
  - 127.0.0.1:11215:1
175
184
 
176
- Finally, to make writing syntactically correct configuration file easier, nutcracker provides a command-line argument -t or --test-conf that can be used to test the YAML configuration file for any syntax error.
185
+ Finally, to make writing a syntactically correct configuration file easier, twemproxy provides a command-line argument -t or --test-conf that can be used to test the YAML configuration file for any syntax error.
177
186
 
178
187
  ## Observability
179
188
 
180
- Observability in nutcracker is through logs and stats.
189
+ Observability in twemproxy is through logs and stats.
181
190
 
182
- Nutcracker exposes stats at the granularity of server pool and servers per pool through the stats monitoring port. The stats are essentially JSON formatted key-value pairs, with the keys corresponding to counter names. By default stats are exposed on port 22222 and aggregated every 30 seconds. Both these values can be configured on program start using the -c or --conf-file and -i or --stats-interval command-line arguments respectively. You can print the description of all stats exported by nutcracker using the -D or --describe-stats command-line argument.
191
+ Twemproxy exposes stats at the granularity of server pool and servers per pool through the stats monitoring port. The stats are essentially JSON formatted key-value pairs, with the keys corresponding to counter names. By default stats are exposed on port 22222 and aggregated every 30 seconds. Both these values can be configured on program start using the -c or --conf-file and -i or --stats-interval command-line arguments respectively. You can print the description of all stats exported by using the -D or --describe-stats command-line argument.
183
192
 
184
193
  $ nutcracker --describe-stats
185
194
 
@@ -205,20 +214,31 @@ Nutcracker exposes stats at the granularity of server pool and servers per pool
205
214
  out_queue "# requests in outgoing queue"
206
215
  out_queue_bytes "current request bytes in outgoing queue"
207
216
 
208
- Logging in nutcracker is only available when nutcracker is built with logging enabled. By default logs are written to stderr. Nutcracker can also be configured to write logs to a specific file through the -o or --output command-line argument. On a running nutcracker, we can turn log levels up and down by sending it SIGTTIN and SIGTTOU signals respectively and reopen log files by sending it SIGHUP signal.
217
+ Logging in twemproxy is only available when twemproxy is built with logging enabled. By default logs are written to stderr. Twemproxy can also be configured to write logs to a specific file through the -o or --output command-line argument. On a running twemproxy, we can turn log levels up and down by sending it SIGTTIN and SIGTTOU signals respectively and reopen log files by sending it SIGHUP signal.
209
218
 
210
219
  ## Pipelining
211
220
 
221
+ Twemproxy enables proxying multiple client connections onto one or few server connections. This architectural setup makes it ideal for pipelining requests and responses and hence saving on the round trip time.
212
222
 
213
- Nutcracker enables proxying multiple client connections onto one or few server connections. This architectural setup makes it ideal for pipelining requests and responses and hence saving on the round trip time.
214
-
215
- For example, if nutcracker is proxying three client connections onto a single server and we get requests - 'get key\r\n', 'set key 0 0 3\r\nval\r\n' and 'delete key\r\n' on these three connections respectively, nutcracker would try to batch these requests and send them as a single message onto the server connection as 'get key\r\nset key 0 0 3\r\nval\r\ndelete key\r\n'.
223
+ For example, if twemproxy is proxying three client connections onto a single server and we get requests - 'get key\r\n', 'set key 0 0 3\r\nval\r\n' and 'delete key\r\n' on these three connections respectively, twemproxy would try to batch these requests and send them as a single message onto the server connection as 'get key\r\nset key 0 0 3\r\nval\r\ndelete key\r\n'.
216
224
 
217
- Pipelining is the reason why nutcracker ends up doing better in terms of throughput even though it introduces an extra hop between the client and server.
225
+ Pipelining is the reason why twemproxy ends up doing better in terms of throughput even though it introduces an extra hop between the client and server.
218
226
 
219
227
  ## Deployment
220
228
 
221
- If you are deploying nutcracker in production, you might consider reading through the [recommendation document](notes/recommendation.md) to understand the parameters you could tune in nutcracker to run it efficiently in the production environment.
229
+ If you are deploying twemproxy in production, you might consider reading through the [recommendation document](notes/recommendation.md) to understand the parameters you could tune in twemproxy to run it efficiently in the production environment.
230
+
231
+ ## Packages
232
+
233
+ ### Ubuntu
234
+
235
+ #### PPA Stable
236
+
237
+ https://launchpad.net/~twemproxy/+archive/ubuntu/stable
238
+
239
+ #### PPA Daily
240
+
241
+ https://launchpad.net/~twemproxy/+archive/ubuntu/daily
222
242
 
223
243
  ## Utils
224
244
  + [nagios checks](https://github.com/wanelo/nagios-checks/blob/master/check_twemproxy)
@@ -231,8 +251,11 @@ If you are deploying nutcracker in production, you might consider reading throug
231
251
  + [sensu-metrics](https://github.com/sensu/sensu-community-plugins/blob/master/plugins/twemproxy/twemproxy-metrics.rb)
232
252
  + [redis-mgr](https://github.com/idning/redis-mgr)
233
253
  + [smitty for twemproxy failover](https://github.com/areina/smitty)
254
+ + [Beholder, a Python agent for twemproxy failover](https://github.com/Serekh/beholder)
255
+ + [chef cookbook](https://supermarket.getchef.com/cookbooks/twemproxy)
256
+ + [twemsentinel] (https://github.com/yak0/twemsentinel)
234
257
 
235
- ## Users
258
+ ## Companies using Twemproxy in Production
236
259
  + [Pinterest](http://pinterest.com/)
237
260
  + [Tumblr](https://www.tumblr.com/)
238
261
  + [Twitter](https://twitter.com/)
@@ -250,6 +273,18 @@ If you are deploying nutcracker in production, you might consider reading throug
250
273
  + [3scale.net](http://3scale.net)
251
274
  + [Ooyala](http://www.ooyala.com)
252
275
  + [Twitch](http://twitch.tv)
276
+ + [Socrata](http://www.socrata.com/)
277
+ + [Hootsuite](http://hootsuite.com/)
278
+ + [Trivago](http://www.trivago.com/)
279
+ + [Machinezone](http://www.machinezone.com)
280
+ + [Flickr](https://www.flickr.com)
281
+ + [Yahoo!](https://www.yahoo.com)
282
+ + [Path](https://path.com)
283
+ + [AOL](http://engineering.aol.com/)
284
+ + [Soysuper](https://soysuper.com/)
285
+ + [Vinted](http://vinted.com/)
286
+ + [Poshmark](https://poshmark.com/)
287
+ + [FanDuel](https://www.fanduel.com/)
253
288
 
254
289
  ## Issues and Support
255
290