memcache 1.2.0 → 1.4.0

package/README.md

@@ -1,4 +1,4 @@
-[<img src="https://jaredwray.com/images/memcache.svg" alt="Memcache Logo" align="center">](https://memcachejs.org)
+[<img src="https://jaredwray.com/images/memcache.svg" width="80%" height="80%" align="center" alt="Memcache Logo" align="center">](https://memcachejs.org)
 
 [![codecov](https://codecov.io/gh/jaredwray/memcache/graph/badge.svg?token=4DUANNWiIE)](https://codecov.io/gh/jaredwray/memcache)
 [![tests](https://github.com/jaredwray/memcache/actions/workflows/tests.yaml/badge.svg)](https://github.com/jaredwray/memcache/actions/workflows/tests.yaml)
@@ -44,6 +44,27 @@ Nodejs Memcache Client
     - [decr(key, value?)](#decrkey-value)
     - [touch(key, exptime)](#touchkey-exptime)
   - [Hook Examples](#hook-examples)
+- [Distribution Algorithms](#distribution-algorithms)
+  - [KetamaHash (Default)](#ketamahash-default)
+  - [ModulaHash](#modulahash)
+  - [Choosing an Algorithm](#choosing-an-algorithm)
+- [Retry Configuration](#retry-configuration)
+  - [Basic Retry Setup](#basic-retry-setup)
+  - [Backoff Strategies](#backoff-strategies)
+  - [Idempotent Safety](#idempotent-safety)
+  - [Methods Without Retry Support](#methods-without-retry-support)
+- [SASL Authentication](#sasl-authentication)
+  - [Enabling SASL Authentication](#enabling-sasl-authentication)
+  - [SASL Options](#sasl-options)
+  - [Per-Node SASL Configuration](#per-node-sasl-configuration)
+  - [Authentication Events](#authentication-events)
+  - [Server Configuration](#server-configuration)
+- [Auto Discovery](#auto-discovery)
+  - [Enabling Auto Discovery](#enabling-auto-discovery)
+  - [Auto Discovery Options](#auto-discovery-options)
+  - [Auto Discovery Events](#auto-discovery-events)
+  - [Legacy Command Support](#legacy-command-support)
+- [IPv6 Support](#ipv6-support)
 - [Contributing](#contributing)
 - [License and Copyright](#license-and-copyright)
 
@@ -179,6 +200,11 @@ const client = new Memcache({
 - `keepAlive?: boolean` - Keep connection alive (default: true)
 - `keepAliveDelay?: number` - Keep alive delay in milliseconds (default: 1000)
 - `hash?: HashProvider` - Hash provider for consistent hashing (default: KetamaHash)
+- `retries?: number` - Number of retry attempts for failed commands (default: 0)
+- `retryDelay?: number` - Base delay in milliseconds between retries (default: 100)
+- `retryBackoff?: RetryBackoffFunction` - Function to calculate backoff delay (default: fixed delay)
+- `retryOnlyIdempotent?: boolean` - Only retry commands marked as idempotent (default: true)
+- `autoDiscover?: AutoDiscoverOptions` - AWS ElastiCache Auto Discovery configuration (see [Auto Discovery](#auto-discovery))
 
 ## Properties
 
@@ -200,6 +226,18 @@ Get or set the keepAlive setting. Updates all existing nodes. Requires `reconnec
 ### `keepAliveDelay: number`
 Get or set the keep alive delay in milliseconds. Updates all existing nodes. Requires `reconnect()` to apply changes.
 
+### `retries: number`
+Get or set the number of retry attempts for failed commands (default: 0).
+
+### `retryDelay: number`
+Get or set the base delay in milliseconds between retry attempts (default: 100).
+
+### `retryBackoff: RetryBackoffFunction`
+Get or set the backoff function for calculating retry delays.
+
+### `retryOnlyIdempotent: boolean`
+Get or set whether retries are restricted to idempotent commands only (default: true).
+
 ## Connection Management
 
 ### `connect(nodeId?: string): Promise<void>`
@@ -373,6 +411,9 @@ client.on('miss', (key) => {
 - `quit` - Emitted when quit command is sent
 - `warn` - Emitted for warning messages
 - `info` - Emitted for informational messages
+- `autoDiscover` - Emitted on initial auto discovery with the cluster config
+- `autoDiscoverUpdate` - Emitted when auto discovery detects a topology change
+- `autoDiscoverError` - Emitted when auto discovery encounters an error
 
 ## Hooks
 
@@ -473,9 +514,534 @@ client.onHook('after:set', async (context) => {
 });
 ```
 
+# Distribution Algorithms
+
+Memcache supports pluggable distribution algorithms to determine how keys are distributed across nodes. You can configure the algorithm using the `hash` option.
+
+## KetamaHash (Default)
+
+KetamaHash uses the Ketama consistent hashing algorithm, which minimizes key redistribution when nodes are added or removed. This is the default and recommended algorithm for production environments with dynamic scaling.
+
+```javascript
+import { Memcache } from 'memcache';
+
+// KetamaHash is used by default
+const client = new Memcache({
+  nodes: ['server1:11211', 'server2:11211', 'server3:11211']
+});
+```
+
+**Characteristics:**
+- Minimal key redistribution (~1/n keys move when adding/removing nodes)
+- Uses virtual nodes for better distribution
+- Supports weighted nodes
+- Best for production environments with dynamic scaling
+
+## ModulaHash
+
+ModulaHash uses a simple modulo-based hashing algorithm (`hash(key) % nodeCount`). This is a simpler algorithm that may redistribute all keys when nodes change.
+
+```javascript
+import { Memcache, ModulaHash } from 'memcache';
+
+// Use ModulaHash for distribution
+const client = new Memcache({
+  nodes: ['server1:11211', 'server2:11211', 'server3:11211'],
+  hash: new ModulaHash()
+});
+
+// With a custom hash algorithm (default is sha1)
+const client2 = new Memcache({
+  nodes: ['server1:11211', 'server2:11211'],
+  hash: new ModulaHash('md5')
+});
+```
+
+**Characteristics:**
+- Simple and fast algorithm
+- All keys may be redistributed when nodes are added or removed
+- Supports weighted nodes (nodes with higher weight appear more in the distribution)
+- Best for fixed-size clusters or testing environments
+
+### Weighted Nodes with ModulaHash
+
+ModulaHash supports weighted nodes, where nodes with higher weights receive proportionally more keys:
+
+```javascript
+import { Memcache, ModulaHash, createNode } from 'memcache';
+
+// Create nodes with different weights
+const node1 = createNode('server1', 11211, { weight: 3 }); // 3x traffic
+const node2 = createNode('server2', 11211, { weight: 1 }); // 1x traffic
+
+const client = new Memcache({
+  nodes: [node1, node2],
+  hash: new ModulaHash()
+});
+
+// server1 will receive approximately 75% of keys
+// server2 will receive approximately 25% of keys
+```
+
+## Choosing an Algorithm
+
+| Feature | KetamaHash | ModulaHash |
+|---------|------------|------------|
+| Key redistribution on node change | Minimal (~1/n keys) | All keys may move |
+| Complexity | Higher (virtual nodes) | Lower (simple modulo) |
+| Performance | Slightly slower | Faster |
+| Best for | Dynamic scaling | Fixed clusters |
+| Weighted nodes | Yes | Yes |
+
+**Use KetamaHash (default) when:**
+- Your cluster size may change dynamically
+- You want to minimize cache invalidation during scaling
+- You're running in production
+
+**Use ModulaHash when:**
+- Your cluster size is fixed
+- You prefer simplicity over minimal redistribution
+- You're in a testing or development environment
+
+# Retry Configuration
+
+The Memcache client supports automatic retry of failed commands with configurable backoff strategies.
+
+## Basic Retry Setup
+
+Enable retries by setting the `retries` option:
+
+```javascript
+import { Memcache } from 'memcache';
+
+const client = new Memcache({
+  nodes: ['localhost:11211'],
+  retries: 3,        // Retry up to 3 times
+  retryDelay: 100    // 100ms between retries
+});
+```
+
+You can also modify retry settings at runtime:
+
+```javascript
+client.retries = 5;
+client.retryDelay = 200;
+```
+
+## Backoff Strategies
+
+The client includes two built-in backoff functions:
+
+### Fixed Delay (Default)
+
+```javascript
+import { Memcache, defaultRetryBackoff } from 'memcache';
+
+const client = new Memcache({
+  retries: 3,
+  retryDelay: 100,
+  retryBackoff: defaultRetryBackoff  // 100ms, 100ms, 100ms
+});
+```
+
+### Exponential Backoff
+
+```javascript
+import { Memcache, exponentialRetryBackoff } from 'memcache';
+
+const client = new Memcache({
+  retries: 3,
+  retryDelay: 100,
+  retryBackoff: exponentialRetryBackoff  // 100ms, 200ms, 400ms
+});
+```
+
+### Custom Backoff Function
+
+You can provide your own backoff function:
+
+```javascript
+const client = new Memcache({
+  retries: 3,
+  retryDelay: 100,
+  retryBackoff: (attempt, baseDelay) => {
+    // Exponential backoff with jitter
+    const delay = baseDelay * Math.pow(2, attempt);
+    return delay + Math.random() * delay * 0.1;
+  }
+});
+```
+
+The backoff function receives:
+- `attempt` - The current attempt number (0-indexed)
+- `baseDelay` - The configured `retryDelay` value
+
+## Idempotent Safety
+
+**Important:** By default, retries are only performed for commands explicitly marked as idempotent. This prevents accidental double-execution of non-idempotent operations like `incr`, `decr`, `append`, and `prepend`.
+
+### Why This Matters
+
+If a network timeout occurs after the server applies a mutation but before the client receives the response, retrying would apply the mutation twice:
+- Counter incremented twice instead of once
+- Data appended twice instead of once
+
+### Safe Usage Patterns
+
+**For read operations (always safe to retry):**
+
+```javascript
+// Mark read operations as idempotent
+await client.execute('get mykey', nodes, { idempotent: true });
+```
+
+**For idempotent writes (safe to retry):**
+
+```javascript
+// SET with the same value is idempotent
+await client.execute('set mykey 0 0 5\r\nhello', nodes, { idempotent: true });
+```
+
+**Disable safety for all commands (use with caution):**
+
+```javascript
+const client = new Memcache({
+  retries: 3,
+  retryOnlyIdempotent: false  // Allow retries for ALL commands
+});
+```
+
+### Behavior Summary
+
+| `retryOnlyIdempotent` | `idempotent` flag | Retries enabled? |
+|-----------------------|-------------------|------------------|
+| `true` (default)      | `false` (default) | No               |
+| `true` (default)      | `true`            | Yes              |
+| `false`               | (any)             | Yes              |
+
+### Methods Without Retry Support
+
+The following methods do not use the retry mechanism and have their own error handling:
+
+- `get()` - Returns `undefined` on failure
+- `gets()` - Returns partial results on node failure
+- `flush()` - Operates directly on nodes
+- `stats()` - Operates directly on nodes
+- `version()` - Operates directly on nodes
+
+To use retries with read operations, use the `execute()` method directly:
+
+```javascript
+const nodes = await client.getNodesByKey('mykey');
+const results = await client.execute('get mykey', nodes, { idempotent: true });
+```
+
+# SASL Authentication
+
+The Memcache client supports SASL (Simple Authentication and Security Layer) authentication using the PLAIN mechanism. This allows you to connect to memcached servers that require authentication.
+
+## Enabling SASL Authentication
+
+```javascript
+import { Memcache } from 'memcache';
+
+const client = new Memcache({
+  nodes: ['localhost:11211'],
+  sasl: {
+    username: 'myuser',
+    password: 'mypassword',
+  },
+});
+
+await client.connect();
+// Client is now authenticated and ready to use
+```
+
+## SASL Options
+
+The `sasl` option accepts an object with the following properties:
+
+- `username: string` - The username for authentication (required)
+- `password: string` - The password for authentication (required)
+- `mechanism?: 'PLAIN'` - The SASL mechanism to use (default: 'PLAIN')
+
+Currently, only the PLAIN mechanism is supported.
+
+## Binary Protocol Methods
+
+**Important:** Memcached servers with SASL enabled (`-S` flag) require the binary protocol for all operations after authentication. The standard text-based methods (`client.get()`, `client.set()`, etc.) will not work on SASL-enabled servers.
+
+Use the `binary*` methods on nodes for SASL-enabled servers:
+
+```javascript
+import { Memcache } from 'memcache';
+
+const client = new Memcache({
+  nodes: ['localhost:11211'],
+  sasl: { username: 'user', password: 'pass' },
+});
+
+await client.connect();
+
+// Access the node directly for binary operations
+const node = client.nodes[0];
+
+// Binary protocol operations
+await node.binarySet('mykey', 'myvalue', 3600);     // Set with 1 hour expiry
+const value = await node.binaryGet('mykey');         // Get value
+await node.binaryDelete('mykey');                    // Delete key
+
+// Other binary operations
+await node.binaryAdd('newkey', 'value');             // Add (only if not exists)
+await node.binaryReplace('existingkey', 'newvalue'); // Replace (only if exists)
+await node.binaryIncr('counter', 1);                 // Increment
+await node.binaryDecr('counter', 1);                 // Decrement
+await node.binaryAppend('mykey', '-suffix');         // Append to value
+await node.binaryPrepend('mykey', 'prefix-');        // Prepend to value
+await node.binaryTouch('mykey', 7200);               // Update expiration
+await node.binaryFlush();                            // Flush all
+const version = await node.binaryVersion();          // Get server version
+const stats = await node.binaryStats();              // Get server stats
+```
+
+## Per-Node SASL Configuration
+
+You can also configure SASL credentials when creating individual nodes:
+
+```javascript
+import { createNode } from 'memcache';
+
+// Create a node with SASL credentials
+const node = createNode('localhost', 11211, {
+  sasl: { username: 'user', password: 'pass' },
+});
+
+// Connect and use binary methods
+await node.connect();
+await node.binarySet('mykey', 'hello');
+const value = await node.binaryGet('mykey');
+```
+
+## Authentication Events
+
+You can listen for authentication events on both nodes and the client:
+
+```javascript
+import { Memcache, MemcacheNode } from 'memcache';
+
+// Node-level events
+const node = new MemcacheNode('localhost', 11211, {
+  sasl: { username: 'user', password: 'pass' },
+});
+
+node.on('authenticated', () => {
+  console.log('Node authenticated successfully');
+});
+
+node.on('error', (error) => {
+  if (error.message.includes('SASL authentication failed')) {
+    console.error('Authentication failed:', error.message);
+  }
+});
+
+await node.connect();
+
+// Client-level events (forwarded from nodes)
+const client = new Memcache({
+  nodes: ['localhost:11211'],
+  sasl: { username: 'user', password: 'pass' },
+});
+
+client.on('authenticated', () => {
+  console.log('Client authenticated');
+});
+
+await client.connect();
+```
+
+### Node Properties
+
+- `node.hasSaslCredentials` - Returns `true` if SASL credentials are configured
+- `node.isAuthenticated` - Returns `true` if the node has successfully authenticated
+
+## Server Configuration
+
+To use SASL authentication, your memcached server must be configured with SASL support:
+
+1. **Build memcached with SASL support** - Ensure memcached was compiled with `--enable-sasl`
+
+2. **Create SASL users** - Use `saslpasswd2` to create users:
+   ```bash
+   saslpasswd2 -a memcached -c username
+   ```
+
+3. **Configure SASL mechanism** - Create `/etc/sasl2/memcached.conf`:
+   ```
+   mech_list: plain
+   ```
+
+4. **Start memcached with SASL** - Use the `-S` flag:
+   ```bash
+   memcached -S -m 64 -p 11211
+   ```
+
+For more details, see the [memcached SASL documentation](https://github.com/memcached/memcached/wiki/SASLHowto).
+
+# Auto Discovery
+
+The Memcache client supports AWS ElastiCache Auto Discovery, which automatically detects cluster topology changes and adds or removes nodes as needed. When enabled, the client connects to a configuration endpoint, retrieves the current list of cache nodes, and periodically polls for changes.
+
+## Enabling Auto Discovery
+
+```javascript
+import { Memcache } from 'memcache';
+
+const client = new Memcache({
+  nodes: [],
+  autoDiscover: {
+    enabled: true,
+    configEndpoint: 'my-cluster.cfg.use1.cache.amazonaws.com:11211',
+  },
+});
+
+await client.connect();
+// The client automatically discovers and connects to all cluster nodes
+```
+
+If you omit `configEndpoint`, the first node in the `nodes` array is used as the configuration endpoint:
+
+```javascript
+const client = new Memcache({
+  nodes: ['my-cluster.cfg.use1.cache.amazonaws.com:11211'],
+  autoDiscover: {
+    enabled: true,
+  },
+});
+```
+
+## Auto Discovery Options
+
+The `autoDiscover` option accepts an object with the following properties:
+
+- `enabled: boolean` - Enable auto discovery of cluster nodes (required)
+- `pollingInterval?: number` - How often to poll for topology changes, in milliseconds (default: 60000)
+- `configEndpoint?: string` - The configuration endpoint to use for discovery. This is typically the `.cfg` endpoint from ElastiCache. If not specified, the first node in the `nodes` array will be used
+- `useLegacyCommand?: boolean` - Use the legacy `get AmazonElastiCache:cluster` command instead of `config get cluster` (default: false)
+
+## Auto Discovery Events
+
+The client emits events during the auto discovery lifecycle:
+
+```javascript
+const client = new Memcache({
+  nodes: [],
+  autoDiscover: {
+    enabled: true,
+    configEndpoint: 'my-cluster.cfg.use1.cache.amazonaws.com:11211',
+  },
+});
+
+// Emitted on initial discovery with the full cluster config
+client.on('autoDiscover', (config) => {
+  console.log('Discovered nodes:', config.nodes);
+  console.log('Config version:', config.version);
+});
+
+// Emitted when polling detects a topology change
+client.on('autoDiscoverUpdate', (config) => {
+  console.log('Cluster topology changed:', config.nodes);
+});
+
+// Emitted when discovery encounters an error (non-fatal, retries on next poll)
+client.on('autoDiscoverError', (error) => {
+  console.error('Discovery error:', error.message);
+});
+
+await client.connect();
+```
+
+## Legacy Command Support
+
+For ElastiCache engine versions older than 1.4.14, use the legacy discovery command:
+
+```javascript
+const client = new Memcache({
+  nodes: [],
+  autoDiscover: {
+    enabled: true,
+    configEndpoint: 'my-cluster.cfg.use1.cache.amazonaws.com:11211',
+    useLegacyCommand: true, // Uses 'get AmazonElastiCache:cluster' instead of 'config get cluster'
+  },
+});
+```
+
+# IPv6 Support
+
+The Memcache client fully supports IPv6 addresses using standard bracket notation in URIs.
+
+## Connecting to IPv6 Nodes
+
+```javascript
+import { Memcache } from 'memcache';
+
+// IPv6 loopback
+const client = new Memcache('[::1]:11211');
+
+// Multiple IPv6 nodes
+const client = new Memcache({
+  nodes: [
+    '[::1]:11211',
+    '[2001:db8::1]:11211',
+    'memcache://[2001:db8::2]:11212',
+  ],
+});
+
+await client.connect();
+```
+
+## IPv6 in Auto Discovery
+
+When auto discovery returns IPv6 node addresses, the client automatically brackets them for correct URI handling:
+
+```javascript
+const client = new Memcache({
+  nodes: [],
+  autoDiscover: {
+    enabled: true,
+    configEndpoint: '[2001:db8::1]:11211',
+  },
+});
+
+await client.connect();
+// Discovered IPv6 nodes are added as [host]:port automatically
+```
+
+## IPv6 Node IDs
+
+Node IDs for IPv6 addresses use bracket notation to avoid ambiguity:
+
+```javascript
+const client = new Memcache({
+  nodes: ['[::1]:11211', '[2001:db8::1]:11212'],
+});
+
+console.log(client.nodeIds);
+// ['[::1]:11211', '[2001:db8::1]:11212']
+```
+
+# Benchmarks
+
+These are provided to show a simple benchmark against current libraries. This is not robust but it is something we update regularly to make sure we are keeping performant.
+
+|             name             |  summary  |  ops/sec  |  time/op  |  margin  |  samples  |
+|------------------------------|:---------:|----------:|----------:|:--------:|----------:|
+|  memcache set/get (v1.4.0)   |    🥇     |       3K  |    350µs  |  ±0.19%  |      10K  |
+|  memcached set/get (v2.2.2)  |   -2.9%   |       3K  |    361µs  |  ±0.16%  |      10K  |
+|  memjs set/get (v1.3.2)      |   -12%    |       3K  |    398µs  |  ±0.17%  |      10K  |
+
 # Contributing
 
-Please read our [Contributing Guidelines](./CONTRIBUTING.md) and also our [Code of Conduct](./CODE_OF_CONDUCT.md). 
+Please read our [Contributing Guidelines](./CONTRIBUTING.md) and also our [Code of Conduct](./CODE_OF_CONDUCT.md).
 
 # License and Copyright