nutcracker 0.2.4.1 → 0.2.4.2
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +28 -10
- data/Rakefile +5 -19
- data/bin/nutcracker +6 -1
- data/ext/nutcracker/ChangeLog +9 -0
- data/ext/nutcracker/Makefile.in +54 -29
- data/ext/nutcracker/README.md +13 -11
- data/ext/nutcracker/aclocal.m4 +46 -26
- data/ext/nutcracker/config/config.guess +209 -240
- data/ext/nutcracker/config/config.sub +157 -70
- data/ext/nutcracker/config/depcomp +66 -8
- data/ext/nutcracker/config/install-sh +18 -11
- data/ext/nutcracker/config/ltmain.sh +2632 -1384
- data/ext/nutcracker/config/missing +4 -49
- data/ext/nutcracker/configure +2866 -2118
- data/ext/nutcracker/configure.ac +1 -1
- data/ext/nutcracker/contrib/Makefile.in +17 -10
- data/ext/nutcracker/m4/libtool.m4 +1437 -812
- data/ext/nutcracker/m4/ltoptions.m4 +24 -8
- data/ext/nutcracker/m4/ltversion.m4 +6 -6
- data/ext/nutcracker/m4/lt~obsolete.m4 +9 -3
- data/ext/nutcracker/notes/recommendation.md +21 -2
- data/ext/nutcracker/notes/redis.md +9 -9
- data/ext/nutcracker/scripts/redis-check.sh +9 -0
- data/ext/nutcracker/src/Makefile.in +18 -11
- data/ext/nutcracker/src/hashkit/Makefile.am +1 -0
- data/ext/nutcracker/src/hashkit/Makefile.in +23 -13
- data/ext/nutcracker/src/hashkit/nc_crc16.c +66 -0
- data/ext/nutcracker/src/hashkit/nc_hashkit.h +2 -0
- data/ext/nutcracker/src/hashkit/nc_modula.c +18 -6
- data/ext/nutcracker/src/nc_conf.c +14 -35
- data/ext/nutcracker/src/nc_conf.h +1 -1
- data/ext/nutcracker/src/nc_message.h +2 -0
- data/ext/nutcracker/src/nc_server.c +9 -7
- data/ext/nutcracker/src/proto/Makefile.in +16 -9
- data/ext/nutcracker/src/proto/nc_redis.c +17 -4
- data/lib/nutcracker/version.rb +1 -1
- data/lib/nutcracker.rb +60 -2
- metadata +3 -2
data/README.md
CHANGED
@@ -7,18 +7,17 @@ For now this repository only contains a Rakefile for building new `nutcracker` g
|
|
7
7
|
## Motivation
|
8
8
|
The main motivation here is to take the advantages of working with Bundler's dependencies management and to be able to embed Twitter's [Nutcracker](https://github.com/twitter/twemproxy) as a dependency to any Ruby project, this allow you to create small-configuration-only-apps tied to specific version of Nutcracker as I show in the example bellow.
|
9
9
|
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
Or if you are using a `Gemfile`
|
15
|
-
|
16
|
-
```ruby
|
17
|
-
source :rubygems
|
18
|
-
|
10
|
+
### Installation
|
11
|
+
Add this line to your application's Gemfile:
|
12
|
+
```
|
19
13
|
gem 'nutcracker'
|
20
14
|
```
|
21
15
|
|
16
|
+
And then execute:
|
17
|
+
```
|
18
|
+
$ bundle install
|
19
|
+
```
|
20
|
+
|
22
21
|
after the gem was successfully installed, the `nutcracker` executable should be available
|
23
22
|
```
|
24
23
|
[root@somewhere ~]# nutcracker --help
|
@@ -44,9 +43,28 @@ Options:
|
|
44
43
|
-m, --mbuf-size=N : set size of mbuf chunk in bytes (default: 16384 bytes)
|
45
44
|
|
46
45
|
```
|
46
|
+
|
47
|
+
### Ruby Wrapper
|
48
|
+
a simple ruby wrapper is also included
|
49
|
+
|
50
|
+
```ruby
|
51
|
+
require 'nutcracker'
|
52
|
+
|
53
|
+
nutcracker = Nutcracker.start(config_file: 'cluster.yaml')
|
54
|
+
nutcracker.running? # => true
|
55
|
+
|
56
|
+
nutcracker.stats # => {"source"=>"local", "version"=>"0.2.4", "uptime"=>6...}
|
57
|
+
|
58
|
+
nutcracker.stop
|
59
|
+
nutcracker.running? # => false
|
60
|
+
|
61
|
+
nutcracker.start
|
62
|
+
nutcracker.join # wait for server to exit
|
63
|
+
```
|
64
|
+
|
47
65
|
## Wanna build a new version?
|
48
66
|
* Set the version @ `lib/nutcracker/version.rb` ( [Available Versions](https://code.google.com/p/twemproxy/downloads/list) )
|
49
67
|
* run the `rake build` command
|
50
68
|
* look for `nutcracker-X.Y.Z` gem under the pkg folder
|
51
69
|
|
52
|
-
> for more details like licensing etc, please look @ [Nutcracker](https://github.com/twitter/twemproxy)
|
70
|
+
> for more details like licensing etc, please look @ [Nutcracker](https://github.com/twitter/twemproxy)
|
data/Rakefile
CHANGED
@@ -3,31 +3,17 @@ require 'nutcracker'
|
|
3
3
|
require 'rake'
|
4
4
|
require 'rubygems/package_task'
|
5
5
|
|
6
|
-
Nutcracker::GemSpec =
|
7
|
-
|
8
|
-
|
9
|
-
s.platform = Gem::Platform::RUBY
|
10
|
-
s.summary = "Twitter's Nutcraker Gem Wrapper"
|
11
|
-
s.description = "Gem/Bundler benefits for Twitter's Nutcraker C app"
|
12
|
-
s.author = "Eran Barak Levi"
|
13
|
-
s.email = "eran@kontera.com"
|
14
|
-
s.homepage = 'http://www.kontera.com'
|
15
|
-
s.required_ruby_version = '>= 1.8.5'
|
16
|
-
s.rubyforge_project = "ruby-nutcracker"
|
17
|
-
s.files = %w(README.md Rakefile) + Dir.glob("{bin,lib,ext}/**/*")
|
18
|
-
s.require_path = "lib"
|
19
|
-
s.extensions = ['ext/nutcracker/extconf.rb']
|
20
|
-
s.executables = ['nutcracker']
|
21
|
-
s.require_paths = ['lib']
|
22
|
-
end
|
6
|
+
Nutcracker::GemSpec = eval File.read 'nutcracker.gemspec'
|
7
|
+
|
8
|
+
sversion = Nutcracker.version.split(".")[0..2].join(".")
|
23
9
|
|
24
10
|
task :download do
|
25
|
-
"nutcracker-#{
|
11
|
+
"nutcracker-#{sversion}.tar.gz".tap do |tarball|
|
26
12
|
sh "mkdir ext" unless File.directory? "ext"
|
27
13
|
sh "rm -rf ext/nutcracker"
|
28
14
|
sh "wget https://twemproxy.googlecode.com/files/#{tarball}"
|
29
15
|
sh "tar -zxvf #{tarball}"
|
30
|
-
sh "mv nutcracker-#{
|
16
|
+
sh "mv nutcracker-#{sversion} ext/nutcracker"
|
31
17
|
File.open("ext/nutcracker/extconf.rb",'w') do |file|
|
32
18
|
file.puts %q{
|
33
19
|
raise "no support for #{RUBY_PLATFORM}" if RUBY_PLATFORM =~ /darwin|mswin|mingw/
|
data/bin/nutcracker
CHANGED
data/ext/nutcracker/ChangeLog
CHANGED
@@ -1,3 +1,12 @@
|
|
1
|
+
2013-23-04 Manju Rajashekhar <manj@cs.stanford.edu>
|
2
|
+
* twemproxy: version 0.2.4 release
|
3
|
+
redis keys must be less than mbuf_data_size() in length (fifsky)
|
4
|
+
Adds support for DUMP/RESTORE commands in Redis (remotezygote)
|
5
|
+
Use of the weight value in the modula distribution (mezzatto)
|
6
|
+
Add support to unix socket connections to servers (mezzatto)
|
7
|
+
only check for duplicate server name and not 'host:port:weight' when 'name' is configured
|
8
|
+
crc16 hash support added (mezzatto)
|
9
|
+
|
1
10
|
2013-31-01 Manju Rajashekhar <manj@twitter.com>
|
2
11
|
* twemproxy: version 0.2.3 release
|
3
12
|
RPOPLPUSH, SDIFF, SDIFFSTORE, SINTER, SINTERSTORE, SMOVE, SUNION, SUNIONSTORE, ZINTERSTORE, and ZUNIONSTORE support (dcartoon)
|
data/ext/nutcracker/Makefile.in
CHANGED
@@ -1,9 +1,9 @@
|
|
1
|
-
# Makefile.in generated by automake 1.11 from Makefile.am.
|
1
|
+
# Makefile.in generated by automake 1.11.3 from Makefile.am.
|
2
2
|
# @configure_input@
|
3
3
|
|
4
4
|
# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002,
|
5
|
-
# 2003, 2004, 2005, 2006, 2007, 2008, 2009
|
6
|
-
# Inc.
|
5
|
+
# 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software
|
6
|
+
# Foundation, Inc.
|
7
7
|
# This Makefile.in is free software; the Free Software Foundation
|
8
8
|
# gives unlimited permission to copy and/or distribute it,
|
9
9
|
# with or without modifications, as long as this notice is preserved.
|
@@ -73,9 +73,11 @@ DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST)
|
|
73
73
|
distdir = $(PACKAGE)-$(VERSION)
|
74
74
|
top_distdir = $(distdir)
|
75
75
|
am__remove_distdir = \
|
76
|
-
|
77
|
-
|
78
|
-
|
76
|
+
if test -d "$(distdir)"; then \
|
77
|
+
find "$(distdir)" -type d ! -perm -200 -exec chmod u+w {} ';' \
|
78
|
+
&& rm -rf "$(distdir)" \
|
79
|
+
|| { sleep 5 && rm -rf "$(distdir)"; }; \
|
80
|
+
else :; fi
|
79
81
|
am__relativize = \
|
80
82
|
dir0=`pwd`; \
|
81
83
|
sed_first='s,^\([^/]*\)/.*$$,\1,'; \
|
@@ -104,6 +106,8 @@ am__relativize = \
|
|
104
106
|
DIST_ARCHIVES = $(distdir).tar.gz
|
105
107
|
GZIP_ENV = --best
|
106
108
|
distuninstallcheck_listfiles = find . -type f -print
|
109
|
+
am__distuninstallcheck_listfiles = $(distuninstallcheck_listfiles) \
|
110
|
+
| sed 's|^\./|$(prefix)/|' | grep -v '$(infodir)/dir$$'
|
107
111
|
distcleancheck_listfiles = find . -type f -print
|
108
112
|
ACLOCAL = @ACLOCAL@
|
109
113
|
AMTAR = @AMTAR@
|
@@ -124,6 +128,7 @@ CXXFLAGS = @CXXFLAGS@
|
|
124
128
|
CYGPATH_W = @CYGPATH_W@
|
125
129
|
DEFS = @DEFS@
|
126
130
|
DEPDIR = @DEPDIR@
|
131
|
+
DLLTOOL = @DLLTOOL@
|
127
132
|
DSYMUTIL = @DSYMUTIL@
|
128
133
|
DUMPBIN = @DUMPBIN@
|
129
134
|
ECHO_C = @ECHO_C@
|
@@ -147,6 +152,7 @@ LIPO = @LIPO@
|
|
147
152
|
LN_S = @LN_S@
|
148
153
|
LTLIBOBJS = @LTLIBOBJS@
|
149
154
|
MAKEINFO = @MAKEINFO@
|
155
|
+
MANIFEST_TOOL = @MANIFEST_TOOL@
|
150
156
|
MKDIR_P = @MKDIR_P@
|
151
157
|
NM = @NM@
|
152
158
|
NMEDIT = @NMEDIT@
|
@@ -172,6 +178,7 @@ abs_builddir = @abs_builddir@
|
|
172
178
|
abs_srcdir = @abs_srcdir@
|
173
179
|
abs_top_builddir = @abs_top_builddir@
|
174
180
|
abs_top_srcdir = @abs_top_srcdir@
|
181
|
+
ac_ct_AR = @ac_ct_AR@
|
175
182
|
ac_ct_CC = @ac_ct_CC@
|
176
183
|
ac_ct_CXX = @ac_ct_CXX@
|
177
184
|
ac_ct_DUMPBIN = @ac_ct_DUMPBIN@
|
@@ -205,7 +212,6 @@ libdir = @libdir@
|
|
205
212
|
libexecdir = @libexecdir@
|
206
213
|
localedir = @localedir@
|
207
214
|
localstatedir = @localstatedir@
|
208
|
-
lt_ECHO = @lt_ECHO@
|
209
215
|
mandir = @mandir@
|
210
216
|
mkdir_p = @mkdir_p@
|
211
217
|
oldincludedir = @oldincludedir@
|
@@ -230,7 +236,7 @@ all: config.h
|
|
230
236
|
$(MAKE) $(AM_MAKEFLAGS) all-recursive
|
231
237
|
|
232
238
|
.SUFFIXES:
|
233
|
-
am--refresh:
|
239
|
+
am--refresh: Makefile
|
234
240
|
@:
|
235
241
|
$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps)
|
236
242
|
@for dep in $?; do \
|
@@ -266,10 +272,8 @@ $(ACLOCAL_M4): $(am__aclocal_m4_deps)
|
|
266
272
|
$(am__aclocal_m4_deps):
|
267
273
|
|
268
274
|
config.h: stamp-h1
|
269
|
-
@if test ! -f $@; then
|
270
|
-
|
271
|
-
$(MAKE) $(AM_MAKEFLAGS) stamp-h1; \
|
272
|
-
else :; fi
|
275
|
+
@if test ! -f $@; then rm -f stamp-h1; else :; fi
|
276
|
+
@if test ! -f $@; then $(MAKE) $(AM_MAKEFLAGS) stamp-h1; else :; fi
|
273
277
|
|
274
278
|
stamp-h1: $(srcdir)/config.h.in $(top_builddir)/config.status
|
275
279
|
@rm -f stamp-h1
|
@@ -298,7 +302,7 @@ distclean-libtool:
|
|
298
302
|
# (which will cause the Makefiles to be regenerated when you run `make');
|
299
303
|
# (2) otherwise, pass the desired values on the `make' command line.
|
300
304
|
$(RECURSIVE_TARGETS):
|
301
|
-
@failcom='exit 1'; \
|
305
|
+
@fail= failcom='exit 1'; \
|
302
306
|
for f in x $$MAKEFLAGS; do \
|
303
307
|
case $$f in \
|
304
308
|
*=* | --[!k]*);; \
|
@@ -323,7 +327,7 @@ $(RECURSIVE_TARGETS):
|
|
323
327
|
fi; test -z "$$fail"
|
324
328
|
|
325
329
|
$(RECURSIVE_CLEAN_TARGETS):
|
326
|
-
@failcom='exit 1'; \
|
330
|
+
@fail= failcom='exit 1'; \
|
327
331
|
for f in x $$MAKEFLAGS; do \
|
328
332
|
case $$f in \
|
329
333
|
*=* | --[!k]*);; \
|
@@ -487,7 +491,8 @@ distdir: $(DISTFILES)
|
|
487
491
|
fi; \
|
488
492
|
done
|
489
493
|
-test -n "$(am__skip_mode_fix)" \
|
490
|
-
|| find "$(distdir)" -type d ! -perm -
|
494
|
+
|| find "$(distdir)" -type d ! -perm -755 \
|
495
|
+
-exec chmod u+rwx,go+rx {} \; -o \
|
491
496
|
! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
|
492
497
|
! -type d ! -perm -400 -exec chmod a+r {} \; -o \
|
493
498
|
! -type d ! -perm -444 -exec $(install_sh) -c -m a+r {} {} \; \
|
@@ -497,7 +502,11 @@ dist-gzip: distdir
|
|
497
502
|
$(am__remove_distdir)
|
498
503
|
|
499
504
|
dist-bzip2: distdir
|
500
|
-
tardir=$(distdir) && $(am__tar) | bzip2 -
|
505
|
+
tardir=$(distdir) && $(am__tar) | BZIP2=$${BZIP2--9} bzip2 -c >$(distdir).tar.bz2
|
506
|
+
$(am__remove_distdir)
|
507
|
+
|
508
|
+
dist-lzip: distdir
|
509
|
+
tardir=$(distdir) && $(am__tar) | lzip -c $${LZIP_OPT--9} >$(distdir).tar.lz
|
501
510
|
$(am__remove_distdir)
|
502
511
|
|
503
512
|
dist-lzma: distdir
|
@@ -505,7 +514,7 @@ dist-lzma: distdir
|
|
505
514
|
$(am__remove_distdir)
|
506
515
|
|
507
516
|
dist-xz: distdir
|
508
|
-
tardir=$(distdir) && $(am__tar) | xz -c >$(distdir).tar.xz
|
517
|
+
tardir=$(distdir) && $(am__tar) | XZ_OPT=$${XZ_OPT--e} xz -c >$(distdir).tar.xz
|
509
518
|
$(am__remove_distdir)
|
510
519
|
|
511
520
|
dist-tarZ: distdir
|
@@ -531,17 +540,19 @@ dist dist-all: distdir
|
|
531
540
|
distcheck: dist
|
532
541
|
case '$(DIST_ARCHIVES)' in \
|
533
542
|
*.tar.gz*) \
|
534
|
-
GZIP=$(GZIP_ENV)
|
543
|
+
GZIP=$(GZIP_ENV) gzip -dc $(distdir).tar.gz | $(am__untar) ;;\
|
535
544
|
*.tar.bz2*) \
|
536
|
-
|
545
|
+
bzip2 -dc $(distdir).tar.bz2 | $(am__untar) ;;\
|
537
546
|
*.tar.lzma*) \
|
538
|
-
|
547
|
+
lzma -dc $(distdir).tar.lzma | $(am__untar) ;;\
|
548
|
+
*.tar.lz*) \
|
549
|
+
lzip -dc $(distdir).tar.lz | $(am__untar) ;;\
|
539
550
|
*.tar.xz*) \
|
540
551
|
xz -dc $(distdir).tar.xz | $(am__untar) ;;\
|
541
552
|
*.tar.Z*) \
|
542
553
|
uncompress -c $(distdir).tar.Z | $(am__untar) ;;\
|
543
554
|
*.shar.gz*) \
|
544
|
-
GZIP=$(GZIP_ENV)
|
555
|
+
GZIP=$(GZIP_ENV) gzip -dc $(distdir).shar.gz | unshar ;;\
|
545
556
|
*.zip*) \
|
546
557
|
unzip $(distdir).zip ;;\
|
547
558
|
esac
|
@@ -555,6 +566,7 @@ distcheck: dist
|
|
555
566
|
&& am__cwd=`pwd` \
|
556
567
|
&& $(am__cd) $(distdir)/_build \
|
557
568
|
&& ../configure --srcdir=.. --prefix="$$dc_install_base" \
|
569
|
+
$(AM_DISTCHECK_CONFIGURE_FLAGS) \
|
558
570
|
$(DISTCHECK_CONFIGURE_FLAGS) \
|
559
571
|
&& $(MAKE) $(AM_MAKEFLAGS) \
|
560
572
|
&& $(MAKE) $(AM_MAKEFLAGS) dvi \
|
@@ -583,8 +595,16 @@ distcheck: dist
|
|
583
595
|
list='$(DIST_ARCHIVES)'; for i in $$list; do echo $$i; done) | \
|
584
596
|
sed -e 1h -e 1s/./=/g -e 1p -e 1x -e '$$p' -e '$$x'
|
585
597
|
distuninstallcheck:
|
586
|
-
|
587
|
-
|
598
|
+
@test -n '$(distuninstallcheck_dir)' || { \
|
599
|
+
echo 'ERROR: trying to run $@ with an empty' \
|
600
|
+
'$$(distuninstallcheck_dir)' >&2; \
|
601
|
+
exit 1; \
|
602
|
+
}; \
|
603
|
+
$(am__cd) '$(distuninstallcheck_dir)' || { \
|
604
|
+
echo 'ERROR: cannot chdir into $(distuninstallcheck_dir)' >&2; \
|
605
|
+
exit 1; \
|
606
|
+
}; \
|
607
|
+
test `$(am__distuninstallcheck_listfiles) | wc -l` -eq 0 \
|
588
608
|
|| { echo "ERROR: files left after uninstall:" ; \
|
589
609
|
if test -n "$(DESTDIR)"; then \
|
590
610
|
echo " (check DESTDIR support)"; \
|
@@ -615,10 +635,15 @@ install-am: all-am
|
|
615
635
|
|
616
636
|
installcheck: installcheck-recursive
|
617
637
|
install-strip:
|
618
|
-
|
619
|
-
|
620
|
-
|
621
|
-
|
638
|
+
if test -z '$(STRIP)'; then \
|
639
|
+
$(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \
|
640
|
+
install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \
|
641
|
+
install; \
|
642
|
+
else \
|
643
|
+
$(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \
|
644
|
+
install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \
|
645
|
+
"INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \
|
646
|
+
fi
|
622
647
|
mostlyclean-generic:
|
623
648
|
|
624
649
|
clean-generic:
|
@@ -707,8 +732,8 @@ uninstall-am:
|
|
707
732
|
.PHONY: $(RECURSIVE_CLEAN_TARGETS) $(RECURSIVE_TARGETS) CTAGS GTAGS \
|
708
733
|
all all-am am--refresh check check-am clean clean-generic \
|
709
734
|
clean-libtool ctags ctags-recursive dist dist-all dist-bzip2 \
|
710
|
-
dist-gzip dist-lzma dist-shar dist-tarZ dist-xz
|
711
|
-
distcheck distclean distclean-generic distclean-hdr \
|
735
|
+
dist-gzip dist-lzip dist-lzma dist-shar dist-tarZ dist-xz \
|
736
|
+
dist-zip distcheck distclean distclean-generic distclean-hdr \
|
712
737
|
distclean-libtool distclean-tags distcleancheck distdir \
|
713
738
|
distuninstallcheck dvi dvi-am html html-am info info-am \
|
714
739
|
install install-am install-data install-data-am install-dvi \
|
data/ext/nutcracker/README.md
CHANGED
@@ -35,7 +35,7 @@ To build nutcracker from source with _debug logs enabled_ and _assertions disabl
|
|
35
35
|
+ Supports proxying to multiple servers.
|
36
36
|
+ Supports multiple server pools simultaneously.
|
37
37
|
+ Shard data automatically across multiple servers.
|
38
|
-
+ Implements the complete [memcached ascii](
|
38
|
+
+ Implements the complete [memcached ascii](notes/memcache.txt) and [redis](notes/redis.md) protocol.
|
39
39
|
+ Easy configuration of server pools through a YAML file.
|
40
40
|
+ Supports multiple hashing modes including consistent hashing and distribution.
|
41
41
|
+ Can be configured to disable nodes on failures.
|
@@ -76,6 +76,7 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
|
|
76
76
|
+ **hash**: The name of the hash function. Possible values are:
|
77
77
|
+ one_at_a_time
|
78
78
|
+ md5
|
79
|
+
+ crc16
|
79
80
|
+ crc32
|
80
81
|
+ fnv1_64
|
81
82
|
+ fnv1a_64
|
@@ -84,7 +85,7 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
|
|
84
85
|
+ hsieh
|
85
86
|
+ murmur
|
86
87
|
+ jenkins
|
87
|
-
+ **hash_tag**: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". [Hash tag](
|
88
|
+
+ **hash_tag**: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". [Hash tag](notes/recommendation.md#hash-tags) enable mapping different keys to the same server as long as the part of the key within the tag is the same.
|
88
89
|
+ **distribution**: The key distribution mode. Possible values are:
|
89
90
|
+ ketama
|
90
91
|
+ modula
|
@@ -94,13 +95,13 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
|
|
94
95
|
+ **preconnect**: A boolean value that controls if nutcracker should preconnect to all the servers in this pool on process start. Defaults to false.
|
95
96
|
+ **redis**: A boolean value that controls if a server pool speaks redis or memcached protocol. Defaults to false.
|
96
97
|
+ **server_connections**: The maximum number of connections that can be opened to each server. By default, we open at most 1 server connection.
|
97
|
-
+ **auto_eject_hosts**: A boolean value that controls if server should be ejected temporarily when it fails consecutively server_failure_limit times. Defaults to false.
|
98
|
+
+ **auto_eject_hosts**: A boolean value that controls if server should be ejected temporarily when it fails consecutively server_failure_limit times. See [liveness recommendations](notes/recommendation.md#liveness) for information. Defaults to false.
|
98
99
|
+ **server_retry_timeout**: The timeout value in msec to wait for before retrying on a temporarily ejected server, when auto_eject_host is set to true. Defaults to 30000 msec.
|
99
100
|
+ **server_failure_limit**: The number of conseutive failures on a server that would leads to it being temporarily ejected when auto_eject_host is set to true. Defaults to 2.
|
100
101
|
+ **servers**: A list of server address, port and weight (name:port:weight or ip:port:weight) for this server pool.
|
101
102
|
|
102
103
|
|
103
|
-
For example, the configuration file in [conf/nutcracker.yml](
|
104
|
+
For example, the configuration file in [conf/nutcracker.yml](conf/nutcracker.yml), also shown below, configures 5 server pools with names - _alpha_, _beta_, _gamma_, _delta_ and omega. Clients that intend to send requests to one of the 10 servers in pool delta connect to port 22124 on 127.0.0.1. Clients that intend to send request to one of 2 servers in pool omega connect to unix path /tmp/gamma. Requests sent to pool alpha and omega have no timeout and might require timeout functionality to be implemented on the client side. On the other hand, requests sent to pool beta, gamma and delta timeout after 400 msec, 400 msec and 100 msec respectively when no response is received from the server. Of the 5 server pools, only pools alpha, gamma and delta are configured to use server ejection and hence are resilient to server failures. All the 5 server pools use ketama consistent hashing for key distribution with the key hasher for pools alpha, beta, gamma and delta set to fnv1a_64 while that for pool omega set to hsieh. Also only pool beta uses [nodes names](notes/recommendation.md#node-names-for-consistent-hashing) for consistent hashing, while pool alpha, gamma, delta and omega use 'host:port:weight' for consistent hashing. Finally, only pool alpha and beta can speak redis protocol, while pool gamma, deta and omega speak memcached protocol.
|
104
105
|
|
105
106
|
alpha:
|
106
107
|
listen: 127.0.0.1:22121
|
@@ -112,7 +113,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
|
|
112
113
|
server_failure_limit: 1
|
113
114
|
servers:
|
114
115
|
- 127.0.0.1:6379:1
|
115
|
-
|
116
|
+
|
116
117
|
beta:
|
117
118
|
listen: 127.0.0.1:22122
|
118
119
|
hash: fnv1a_64
|
@@ -126,7 +127,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
|
|
126
127
|
- 127.0.0.1:6381:1 server2
|
127
128
|
- 127.0.0.1:6382:1 server3
|
128
129
|
- 127.0.0.1:6383:1 server4
|
129
|
-
|
130
|
+
|
130
131
|
gamma:
|
131
132
|
listen: 127.0.0.1:22123
|
132
133
|
hash: fnv1a_64
|
@@ -140,7 +141,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
|
|
140
141
|
servers:
|
141
142
|
- 127.0.0.1:11212:1
|
142
143
|
- 127.0.0.1:11213:1
|
143
|
-
|
144
|
+
|
144
145
|
delta:
|
145
146
|
listen: 127.0.0.1:22124
|
146
147
|
hash: fnv1a_64
|
@@ -160,7 +161,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
|
|
160
161
|
- 127.0.0.1:11221:1
|
161
162
|
- 127.0.0.1:11222:1
|
162
163
|
- 127.0.0.1:11223:1
|
163
|
-
|
164
|
+
|
164
165
|
omega:
|
165
166
|
listen: /tmp/gamma
|
166
167
|
hash: hsieh
|
@@ -169,7 +170,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
|
|
169
170
|
servers:
|
170
171
|
- 127.0.0.1:11214:100000
|
171
172
|
- 127.0.0.1:11215:1
|
172
|
-
|
173
|
+
|
173
174
|
Finally, to make writing syntactically correct configuration file easier, nutcracker provides a command-line argument -t or --test-conf that can be used to test the YAML configuration file for any syntax error.
|
174
175
|
|
175
176
|
## Observability
|
@@ -207,7 +208,7 @@ Logging in nutcracker is only available when nutcracker is built with logging en
|
|
207
208
|
## Pipelining
|
208
209
|
|
209
210
|
|
210
|
-
Nutcracker enables proxying multiple client connections onto one or few server connections. This architectural setup makes it ideal for pipelining requests and responses and hence saving on the round trip time.
|
211
|
+
Nutcracker enables proxying multiple client connections onto one or few server connections. This architectural setup makes it ideal for pipelining requests and responses and hence saving on the round trip time.
|
211
212
|
|
212
213
|
For example, if nutcracker is proxing three client connections onto a single server and we get requests - 'get key\r\n', 'set key 0 0 3\r\nval\r\n' and 'delete key\r\n' on these three connections respectively, nutcracker would try to batch these requests and send them as a single message onto the server connection as 'get key\r\nset key 0 0 3\r\nval\r\ndelete key\r\n'.
|
213
214
|
|
@@ -215,13 +216,14 @@ Pipelining is the reason why nutcracker ends up doing better in terms of through
|
|
215
216
|
|
216
217
|
## Deployment
|
217
218
|
|
218
|
-
If you are deploying nutcracker in production, you might consider reading through the [recommendation document](
|
219
|
+
If you are deploying nutcracker in production, you might consider reading through the [recommendation document](notes/recommendation.md) to understand the parameters you could tune in nutcracker to run it efficiently in the production environment.
|
219
220
|
|
220
221
|
## Users
|
221
222
|
+ [Pinterest](http://pinterest.com/)
|
222
223
|
+ [Tumblr](https://www.tumblr.com/)
|
223
224
|
+ [Twitter](https://twitter.com/)
|
224
225
|
+ [Vine](http://vine.co/)
|
226
|
+
+ [Kiip](http://www.kiip.me/)
|
225
227
|
|
226
228
|
## Issues and Support
|
227
229
|
|