vincentchu-handlersocket 0.0.1

data/ext/HandlerSocket-Plugin-for-MySQL/docs-en/configuration-options.en.txt

@@ -0,0 +1,99 @@
+
+-----------------------------------------------------------------
+handlersocket_verbose (default = 10, min = 0, max = 10000)
+
+  Specify the logging verboseness.
+
+-----------------------------------------------------------------
+handlersocket_address (default = '')
+
+  Specify the address to bind. If empty, it binds to 0.0.0.0.
+
+-----------------------------------------------------------------
+handlersocket_port (default = '9998')
+
+  Specify the port to bind. This option is for the listener for
+  read requests. If empty, the listener is disabled.
+
+-----------------------------------------------------------------
+handlersocket_port_wr (default = '9999')
+
+  Specify the port to bind. This option is for the listener for
+  write requests. If empty, the listener is disabled.
+
+-----------------------------------------------------------------
+handlersocket_epoll (default = 1, min = 0, max = 1)
+
+  Specify whether handlersocket uses epoll for I/O multiplexing.
+
+-----------------------------------------------------------------
+handlersocket_threads (default = 16, min = 1, max = 3000)
+
+  Specify the number of handlersocket worker threads. This option
+  is for the listener for read requests. Recommended value is
+  (the number of CPU cores * 2).
+
+-----------------------------------------------------------------
+handlersocket_threads_wr (default = 1, min = 1, max = 3000)
+
+  Specify the number of handlersocket worker threads. This option
+  is for the listener for write requests. Recommended value is 1.
+
+-----------------------------------------------------------------
+handlersocket_timeout (default = 300, min = 30, max = 3600)
+
+  Specify the socket timeout in seconds.
+
+-----------------------------------------------------------------
+handlersocket_backlog (default = 32768, min = 5, max = 1000000)
+
+  Specify the length of the listen backlog.
+
+-----------------------------------------------------------------
+handlersocket_sndbuf (default = 0, min = 0, max = 1677216)
+
+  Specify the maximum socket send buffer in bytes. If 0, the
+  system-wide default value is set.
+
+-----------------------------------------------------------------
+handlersocket_rcvbuf (default = 0, min = 0, max = 1677216)
+
+  Specify the maximum socket receive buffer in bytes. If 0, the
+  system-wide default value is set.
+
+-----------------------------------------------------------------
+handlersocket_readsize (default = 0, min = 0, max = 1677216)
+
+  Specify the minimum length of the handlersocket request buffer.
+  Larger value can make handlersocket faster for large requests,
+  but can consume memory. The default value is possibly 4096.
+
+-----------------------------------------------------------------
+handlersocket_accept_balance (default = 0, min = 0, max = 10000)
+
+  When this option is set to non-zero, handlersocket tries to
+  balance accepted connections among threads. Non-zero is
+  recommended if you use persistent connections (i.e., connection
+  pooling on the client side).
+
+-----------------------------------------------------------------
+handlersocket_wrlock_timeout (default = 12, min = 0, max = 3600)
+
+  Specify the lock timeout in seconds. When a write request is
+  performed, handlersocket acquires an advisory lock named
+  'handlersocket_wr'. This option sets the timeout for the
+  locking.
+
+-----------------------------------------------------------------
+handlersocket_plain_secret (default = '')
+
+  When this option is specified, a plain-text authentication is
+  enabled for the listener for read requests. This option
+  specifies the secret key for the authentication.
+
+-----------------------------------------------------------------
+handlersocket_plain_secret_wr (default = '')
+
+  This option specifies the secret key for the listener for write
+  requests.
+

data/ext/HandlerSocket-Plugin-for-MySQL/docs-en/installation.en.txt

@@ -0,0 +1,92 @@
+1. Building Handlersocket
+
+ Handlersocket mainly consists of libhsclient, handlersocket, and C++/Perl clients. libhsclient is a common library shared from both client and server(plugin). handlersocket is a MySQL daemon plugin.
+ To build Handlersocket, you need both MySQL source code and MySQL binary. It is not required to pre-build MySQL source code, but source itself is needed because Handlersocket depends on MySQL header files that only MySQL source distribution contains. MySQL binary is just a normal MySQL binary distribution. You can use official MySQL binaries provided by Oracle.
+ Since Handlersocket uses daemon plugin interface supported from MySQL 5.1,
+MySQL 5.1 or higher version is required.
+ Please make sure that you use identical MySQL version between MySQL source
+and MySQL binary. Otherwise you might encounter serious problems (i.e. server
+crash, etc). 
+ Here are steps to build Handlersocket.
+
+* Get MySQL source code
+
+* Get MySQL binary
+
+* Build Handlersocket
+  $ ./autogen.sh
+  $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin  --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
+
+ --with-mysql-source refers to the top of MySQL source directory (which
+contains the VERSION file or the configure.in file), --with-mysql-bindir
+refers to where MySQL binary executables (i.e. mysql_config) are located,
+and --with-mysql-plugindir refers to a plugin directory where plugin
+libraries (*.so) are installed. 
+
+  $ make
+  $ sudo make install
+
+ Both libhsclient and the handlersocket plugin will be installed.
+
+ 
+2. Using Handlersocket
+
+Append configuration options for handlersocket to my.cnf.
+
+  [mysqld]
+  loose_handlersocket_port = 9998
+    # the port number to bind to (for read requests)
+  loose_handlersocket_port_wr = 9999
+    # the port number to bind to (for write requests)
+  loose_handlersocket_threads = 16
+    # the number of worker threads (for read requests)
+  loose_handlersocket_threads_wr = 1
+    # the number of worker threads (for write requests)
+  open_files_limit = 65535
+    # to allow handlersocket accept many concurrent
+    # connections, make open_files_limit as large as
+    # possible.
+
+Log in to mysql as root, and execute the following query.
+
+  mysql> install plugin handlersocket soname 'handlersocket.so';
+
+If handlersocket.so is successfully installed, it starts
+accepting connections on port 9998 and 9999. Running
+'show processlist' should show handlersocket worker threads.
+
+-----------------------------------------------------------------
+On the client side, you need to install libhsclient for c++ apps
+and perl-Net-HandlerSocket for perl apps. They do not require
+MySQL to compile.
+
+  $ ./autogen.sh
+  $ ./configure --disable-handlersocket-server
+  $ make
+  $ sudo make install
+  $ cd perl-Net-HandlerSocket
+  $ perl Makefile.PL
+  $ make
+  $ sudo make install
+
+-----------------------------------------------------------------
+Alternatively, you can use the rpm installation. If your OS
+supports rpms, you can use the following commands to build and
+install handlersocket rpm packages.
+
+(Server side, installs HandlerSocket plugin)
+  $ ./autogen.sh
+  $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin  --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin
+  $ make rpm_cli
+  $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
+  $ make rpm_c
+  $ sudo rpm -U dist/RPMS/*/handlersocket*.rpm
+
+(Client side, installs client libraries)
+  $ ./autogen.sh
+  $ ./configure --disable-handlersocket-server
+  $ make rpm_cli
+  $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm
+  $ make rpm_perl
+  $ sudo rpm -U dist/RPMS/*/perl-Net-HandlerSocket*.rpm
+

data/ext/HandlerSocket-Plugin-for-MySQL/docs-en/perl-client.en.txt

@@ -0,0 +1,135 @@
+
+-----------------------------------------------------------------
+To open a connection to the handlersocket plugin, you need to
+create a Net::HandlerSocket object.
+
+  use Net::HandlerSocket;
+  my $args = { host => 'localhost', port => 9998 };
+  my $hs = new Net::HandlerSocket($args);
+
+-----------------------------------------------------------------
+Before executing table operations, you need to open an index to
+work with.
+
+  my $err = $hs->open_index(3, 'database1', 'table1', 'PRIMARY',
+    'f1,f2');
+  die $hs->get_error() if $res->[0] != 0;
+
+The first argument for open_index is an integer value which is
+used to identify an open table, which is only valid within the
+same Net::HandlerSocket object. The 4th argument is the name of
+index to open. If 'PRIMARY' is specified, the primary index is
+open. The 5th argument is a comma-separated list of column names.
+
+-----------------------------------------------------------------
+To read a record from a table using an index, call the
+execute_single method.
+
+  my $res = $hs->execute_single(3, '=', [ 'foo' ], 1, 0);
+  die $hs->get_error() if $res->[0] != 0;
+  shift(@$res);
+
+The first argument must be an integer which has specified as the
+first argument for open_index on the same Net::HandlerSocket
+object. The second argument specifies the search operation. The
+current version of handlersocket supports '=', '>=', '<=', '>',
+and '<'. The 3rd argument specifies the key to find, which must
+an arrayref whose length is equal to or smaller than the number
+of key columns of the index. The 4th and the 5th arguments
+specify the maximum number of records to be retrieved, and the
+number of records skipped before retrieving records. The columns
+to be retrieved are specified by the 5th argument for the
+corresponding open_index call.
+
+The execute_single method always returns an arrayref. The first
+element is the error code, which is 0 when no error is occured.
+The remaining are the field values. If more than one record is
+returned, it is flatten to an 1-dimensional array. For example,
+when 5 records that have 3 columns are returned, you can retrieve
+values using the following code.
+
+  die $hs->get_error() if $res->[0] != 0;
+  shift(@$res);
+  for (my $row = 0; $row < 5; ++$row) {
+    for (my $col = 0; $col < 3; ++$col) {
+      my $value = $res->[$row * 5 + $col];
+      # ...
+    }
+  }
+
+-----------------------------------------------------------------
+To update or delete records, you need to specify more arguments
+for the execute_single method. Note that the Net::HandlerSocket
+object must be connected to a handlersocket worker for write
+operations, which is port 9999 by default.
+(For safety, the port 9998 only allows read operations, and the
+port 9999 allows write operations also. The port 9999 allows
+read operations too, but slower than 9998 because of record
+locking etc.. Port numbers can be changed using the
+'handlersocket_port' and the 'handlersocket_port_wr'
+configuration options of mysqld.)
+
+  my $args = { host => 'localhost', port => 9999 };
+  my $hs = new Net::HandlerSocket($args);
+
+  my $res = $hs->execute_single(3, '=', [ 'bar' ], 1, 0, 'U',
+    [ 'fubar', 'hoge' ]);
+  die $hs->get_error() if $res->[0] != 0;
+  my $num_updated_rows = $res->[1];
+
+  my $res = $hs->execute_single(3, '=', [ 'baz' ], 1, 0, 'D');
+  die $hs->get_error() if $res->[0] != 0;
+  my $num_deleted_rows = $res->[1];
+
+The 6th argument for execute_single specifies the modification
+operation. The current version supports 'U' and 'D'. For the 'U'
+operation, the 7th argument specifies the new value for the row.
+The columns to be modified are specified by the 5th argument for
+the corresponding open_index call. For the 'D' operation, the
+7th argument can be omitted.
+
+-----------------------------------------------------------------
+The execute_single method can be used for inserting records also.
+
+  my $res = $hs->execute_single(3, '+', [ 'foo', 'bar', 'baz' ]);
+  die $hs->get_error() if $res->[0] != 0;
+  my $num_inserted_rows = $res->[1];
+
+The 3rd argument must be an arrayref whose elements correspond to
+the 5th argument for the corresponding open_index call. If there
+is a column which is not appeared in the 5th argument for the
+open_index, the default value for the column is set.
+
+-----------------------------------------------------------------
+Multiple operations can be executed in a single call. Executing
+multiple operations in a single call is much faster than
+executing them separatedly. 
+
+  my $rarr = $hs->execute_multi([
+    [ 0, '>=', [ 'foo' ], 5, 0 ],
+    [ 2, '=', [ 'bar' ], 1, 0 ],
+    [ 4, '<', [ 'baz' ], 10, 5 ],
+  ]);
+  for my $res (@$rarr) {
+    die $hs->get_error() if $res->[0] != 0;
+    shift(@$res);
+    # ...
+  }
+
+-----------------------------------------------------------------
+If handlersocket is configured to authenticate client connections
+(ie., handlersocket_plain_secret or handlersocket_plain_secret_wr
+is set), a client must call 'auth' method before any other
+methods.
+
+  my $res = $hs->auth('password');
+  die $hs->get_error() if $res->[0] != 0;
+
+-----------------------------------------------------------------
+When an error is occured, the first element of the returned
+arrayref becomes a non-zero value. A negative value indicates
+that an I/O error is occured and the Net::HandlerSocket object
+should be disposed. A positive value means that the connection is
+still active and the Net::HandlerSocket object can be reused
+later.
+

data/ext/HandlerSocket-Plugin-for-MySQL/docs-en/protocol.en.txt

@@ -0,0 +1,205 @@
+
+----------------------------------------------------------------------------
+The HandlerSocket protocol
+
+----------------------------------------------------------------------------
+Basic syntax
+
+- The HandlerSocket protocol is line-based. Each line ends with LF(0x0a).
+- Each line consists a concatenation of tokens separated by HT(0x09).
+- A token is either NULL or an encoded string. Note that you need to
+  distinguish NULL from an empty string, as most DBMs does so.
+- NULL is expressed as a single NUL(0x00).
+- An encoded string is a string with the following encoding rules.
+  	- Characters in the range [0x10 - 0xff] are encoded as itselves.
+	- A character in the range [0x00 - 0x0f] is prefixed by 0x01 and
+	  shifted by 0x40. For example, 0x03 is encoded as 0x01 0x43.
+- Note that a string can be empty. A continuation of 0x09 0x09 means that
+  there is an empty string between them. A continuation of 0x09 0x0a means
+  that there is an empty string at the end of the line.
+
+----------------------------------------------------------------------------
+Request and Response
+
+- The HandlerSocket protocol is a simple request/response protocol. After a
+  connection is established, the client side sends a request, and then the
+  server side sends a response.
+- A request/response consists of a single line.
+- Requests can be pipelined; That is, you can send multiple requests (ie.
+  lines) at one time, and receive responses for them at one time.
+
+----------------------------------------------------------------------------
+Opening index
+
+The 'open_index' request has the following syntax.
+
+    P <indexid> <dbname> <tablename> <indexname> <columns> [<fcolumns>]
+
+- <indexid> is a number in decimal.
+- <dbname>, <tablename>, and <indexname> are strings. To open the primary
+  key, use PRIMARY as <indexname>.
+- <columns> is a comma-separated list of column names.
+- <fcolumns> is a comma-separated list of column names. This parameter is
+  optional.
+
+Once an 'open_index' request is issued, the HandlerSocket plugin opens the
+specified index and keep it open until the client connection is closed. Each
+open index is identified by <indexid>. If <indexid> is already open, the old
+open index is closed. You can open the same combination of <dbname>
+<tablename> <indexname> multple times, possibly with different <columns>.
+For efficiency, keep <indexid> small as far as possible.
+
+----------------------------------------------------------------------------
+Getting data
+
+The 'find' request has the following syntax.
+ 
+    <indexid> <op> <vlen> <v1> ... <vn> [LIM] [IN] [FILTER ...]
+
+LIM is a sequence of the following parameters.
+
+    <limit> <offset>
+
+IN is a sequence of the following parameters.
+    
+    @ <icol> <ivlen> <iv1> ... <ivn>
+
+FILETER is a sequence of the following parameters.
+
+    <ftyp> <fop> <fcol> <fval>
+
+- <indexid> is a number. This number must be an <indexid> specified by a
+  'open_index' request executed previously on the same connection.
+- <op> specifies the comparison operation to use. The current version of
+  HandlerSocket supports '=', '>', '>=', '<', and '<='.
+- <vlen> indicates the length of the trailing parameters <v1> ... <vn>. This
+  must be smaller than or equal to the number of index columns specified by
+  the <indexname> parameter of the corresponding 'open_index' request.
+- <v1> ... <vn> specify the index column values to fetch.
+- LIM is optional. <limit> and <offset> are numbers. When omitted, it works
+  as if 1 and 0 are specified. These parameter works like LIMIT of SQL.
+  These values don't include the number of records skipped by a filter.
+- IN is optional. It works like WHERE ... IN syntax of SQL. <icol> must be
+  smaller than the number of index columns specified by the <indexname>
+  parameter of the corresponding 'open_index' request. If IN is specified in
+  a find request, the <icol>-th parameter value of <v1> ...  <vn> is ignored.
+- FILTERs are optional. A FILTER specifies a filter. <ftyp> is either 'F'
+  (filter) or 'W' (while). <fop> specifies the comparison operation to use.
+  <fcol> must be smaller than the number of columns specified by the
+  <fcolumns> parameter of the corresponding 'open_index' request. Multiple
+  filters can be specified, and work as the logical AND of them. The
+  difference of 'F' and 'W' is that, when a record does not meet the
+  specified condition, 'F' simply skips the record, and 'W' stops the loop.
+
+----------------------------------------------------------------------------
+Updating/Deleting data
+
+The 'find_modify' request has the following syntax.
+
+    <indexid> <op> <vlen> <v1> ... <vn> [LIM] [IN] [FILTER ...] MOD
+
+MOD is a sequence of the following parameters.
+
+    <mop> <m1> ... <mk>
+
+- <mop> is 'U' (update), '+' (increment), '-' (decrement), 'D' (delete),
+  'U?', '+?', '-?', or 'D?'. If the '?' suffix is specified, it returns
+  the contents of the records before modification (as if it's a 'find'
+  request), instead of the number of modified records.
+- <m1> ... <mk> specifies the column values to set. The length of <m1> ...
+  <mk> must be smaller than or equal to the length of <columns> specified by
+  the corresponding 'open_index' request. If <mop> is 'D', these parameters
+  are ignored. If <mop> is '+' or '-', values must be numeric. If <mop> is
+  '-' and it attempts to change a column value from negative to positive or
+  positive to negative, the column value is not modified.
+
+----------------------------------------------------------------------------
+Inserting data
+
+The 'insert' request has the following syntax.
+
+    <indexid> + <vlen> <v1> ... <vn>
+
+- <vlen> indicates the length of the trailing parameters <v1> ... <vn>. This
+  must be smaller than or equal to the length of <columns> specified by the
+  corresponding 'open_index' request.
+- <v1> ... <vn> specify the column values to set. For columns not in
+  <columns>, the default values for each column are set.
+
+----------------------------------------------------------------------------
+Authentication
+
+The 'auth' request has the following syntax.
+
+    A <atyp> <akey>
+
+- <atyp> must be '1'
+- An 'auth' request succeeds iff <akey> is the correct secret specified by
+  the 'handlersocket_plain_secret' or 'handlersocket_plain_secret_rw'.
+- If an authentication is enabled for a listener, any other requests on a
+  connection fail before an 'auth' request succeeded on the connection.
+
+----------------------------------------------------------------------------
+Response syntax
+
+HandlerSocket returns a response of the following syntax for each request.
+
+    <errorcode> <numcolumns> <r1> ... <rn>
+
+- <errorcode> indicates whether the request has successfully executed or not.
+  '0' means success. Non-zero means an error.
+- <numcolumns> indicates the number of columns of the result set.
+- <r1> ... <rn> is the result set. The length of <r1> ... <rn> is always a
+  multiple of <numcolumns>. It is possible that <r1> ... <rn> is empty.
+
+If <errorcode> is non-zero, <numcolumns> is always 1 and <r1> indicates a
+human-readable error message, though sometimes <r1> is not provided.
+
+----------------------------------------------------------------------------
+Response for 'open_index'
+
+If 'open_index' is succeeded, HandlerSocket returns a line of the following
+syntax.
+
+    0 1
+
+----------------------------------------------------------------------------
+Response for 'find'
+
+If 'find' is succeeded, HandlerSocket returns a line of the following
+syntax.
+
+   0 <numcolumns> <r1> ... <rn>
+
+- <numcolumns> always equals to the length of <columns> of the corresponding
+  'open_index' request.
+- <r1> ... <rn> is the result set. If N rows are found, the length of <r1>
+  ... <rn> becomes ( <numcolumns> * N ).
+
+----------------------------------------------------------------------------
+Response for 'find_modify'
+
+If 'find_modify' is succeeded, HandlerSocket returns a line of the following
+syntax.
+
+   0 1 <nummod>
+
+- <nummod> is the number of modified rows.
+- As an exception, if the '?' suffix is specified in <mop>, a response has
+  the syntax of a response for 'find' instead.
+
+----------------------------------------------------------------------------
+Response for 'insert'
+
+If 'insert' is succeeded, HanderSocket returns a line of the following
+syntax.
+
+   0 1
+
+----------------------------------------------------------------------------
+Response for 'auth'
+
+If 'auth' is succeeded, HanderSocket returns a line of the following syntax.
+
+   0 1
+