async-redis 0.11.2 → 0.13.0

data/context/subscriptions.md

@@ -0,0 +1,172 @@
+# Subscriptions
+
+This guide explains how to use Redis pub/sub functionality with `async-redis` to publish and subscribe to messages.
+
+Redis pub/sub enables real-time communication between different parts of your application or between different applications. It's perfect for broadcasting notifications, coordinating distributed systems, or building real-time features, provided you don't need reliable messaging.
+
+Common use cases:
+- **Real-time notifications**: Alert users about new messages, updates, or events.
+- **System coordination**: Notify services about configuration changes or cache invalidation.
+- **Live updates**: Push data changes to web interfaces or mobile apps.
+- **Event-driven architecture**: Decouple services through asynchronous messaging.
+
+## Overview
+
+Redis provides 3 pub/sub mechanisms:
+- **SUBSCRIBE**: Subscribe to specific channels by name.
+- **PSUBSCRIBE**: Subscribe to channels matching patterns (e.g., `user.*`).
+- **SSUBSCRIBE**: Sharded subscriptions for cluster environments (better performance).
+
+## Subscribe
+
+The `SUBSCRIBE` command lets you listen for messages on specific channels. This creates a persistent connection that receives messages as they're published.
+
+Here's a simple notification system - first, create a listener:
+
+``` ruby
+require "async"
+require "async/redis"
+
+client = Async::Redis::Client.new
+
+Async do
+	client.subscribe "status.frontend" do |context|
+		puts "Listening for messages on 'status.frontend'..."
+		
+		type, name, message = context.listen
+		
+		puts "Received: #{message}"
+	end
+end
+```
+
+In another part of your application, publish notifications:
+
+``` ruby
+require "async"
+require "async/redis"
+
+client = Async::Redis::Client.new
+
+Async do
+	puts "Publishing message..."
+	client.publish "status.frontend", "good"
+	puts "Message sent!"
+end
+```
+
+To see pub/sub in action, you can run the listener in one terminal and the publisher in another. The listener will receive any messages sent by the publisher to the `status.frontend` channel:
+
+```bash
+$ ruby listener.rb
+Listening for messages on 'status.frontend'...
+Received: good
+```
+
+### Error Handling
+
+Subscriptions are at-most-once delivery. In addition, subscriptions are stateful, meaning that they maintain their own internal state and can be affected by network issues or server restarts. In order to improve resilience, it's important to implement error handling and reconnection logic.
+
+```ruby
+require "async"
+require "async/redis"
+
+client = Async::Redis::Client.new
+
+Async do
+	client.subscribe "status.frontend" do |context|
+		puts "Listening for messages on 'status.frontend'..."
+		
+		context.each do |type, name, message|
+			puts "Received: #{message}"
+		end
+	end
+rescue => error
+	Console.warn(self, "Subscription failed", error)
+	sleep 1
+	retry
+end
+```
+
+## Pattern Subscribe
+
+When you need to listen to multiple related channels, patterns save you from subscribing to each channel individually. This is perfect for monitoring all user activities or all events from a specific service.
+
+For example, to monitor all user-related events (`user.login`, `user.logout`, `user.signup`):
+
+``` ruby
+require "async"
+require "async/redis"
+
+endpoint = Async::Redis.local_endpoint
+client = Async::Redis::Client.new(endpoint)
+
+Async do
+	client.psubscribe "status.*" do |context|
+		puts "Listening for messages on 'status.*'..."
+		
+		type, pattern, name, message = context.listen
+		
+		puts "Received: #{message}"
+	end
+end
+```
+
+Note that an extra field, `pattern` is returned when using `PSUBSCRIBE`. This field indicates the pattern that was matched for the incoming message. This can be useful for logging or debugging purposes, as it allows you to see which pattern triggered the message delivery.
+
+## Shard Subscribe
+
+If you are working with a clustered environment, you can improve performance by limiting the scope of your subscriptions to specific shards. This can help reduce the amount of data that needs to be sent between shards and improve overall throughput.
+
+To use sharded subscriptions, use a cluster client which supports sharded pub/sub:
+
+``` ruby
+require "async"
+require "async/redis"
+
+# endpoints = ...
+cluster_client = Async::Redis::ClusterClient.new(endpoints)
+
+Async do
+	cluster_client.subscribe "status.frontend" do |context|
+		puts "Listening for messages on 'status.frontend'..."
+		
+		type, name, message = context.listen
+		
+		puts "Received: #{message}"
+	end
+end
+```
+
+``` ruby
+require "async"
+require "async/redis"
+
+# endpoints = ...
+cluster_client = Async::Redis::ClusterClient.new(endpoints)
+
+Async do
+	puts "Publishing message..."
+	cluster_client.publish("status.frontend", "good")
+	puts "Message sent!"
+end
+```
+
+### Clustered Subscriptions
+
+While general `PUBLISH` and `SUBSCRIBE` will work on a cluster, they are less efficient as they require inter-shard communication. By default, the {ruby Async::Redis::ClusterClient} subscription mechanism defaults to `SSUBSCRIBE` and `SPUBLISH`, which are optimized for sharded environments. However, if using multiple subscriptions, internally, several connections will be made to the relevant shards, which increases the complexity.
+
+#### Cluster Topology Changes and Subscription Invalidation
+
+If the cluster is re-configured (e.g. adding or removing nodes, resharding), the subscription state may need to be re-established to account for the new topology. During this process, messages may be lost. This is expected as subscriptions are stateless.
+
+**Important**: When any individual shard subscription fails (due to resharding, node failures, or network issues), the entire cluster subscription is invalidated and will stop delivering messages. This design ensures consistency and prevents partial subscription states that could lead to missed messages on some shards.
+
+Common scenarios that trigger subscription invalidation:
+
+- **Resharding operations**: When slots are migrated between nodes (`MOVED` errors)
+- **Node failures**: When Redis nodes become unavailable
+- **Network partitions**: When connections to specific shards are lost
+- **Cluster reconfiguration**: When the cluster topology changes
+
+Applications should be prepared to handle subscription failures and implement appropriate retry strategies.

data/context/transactions-and-pipelines.md

@@ -0,0 +1,197 @@
+# Transactions and Pipelines
+
+This guide explains how to use Redis transactions and pipelines with `async-redis` for atomic operations and improved performance.
+
+By default, each command (e.g. `GET key`) acquires a connection from the client, runs the command, returns the result and releases the connection back to the client's connection pool. This may be inefficient for some use cases, and so there are several ways to group together operations that run on the same connection.
+
+## Transactions (MULTI/EXEC)
+
+Transactions ensure that multiple Redis commands execute atomically - either all commands succeed, or none of them do. This is crucial when you need to maintain data consistency, such as transferring money between accounts or updating related fields together.
+
+Use transactions when you need:
+- **Atomic updates**: Multiple operations that must all succeed or all fail.
+- **Data consistency**: Keeping related data in sync across multiple keys.
+- **Preventing partial updates**: Avoiding situations where only some of your changes are applied.
+
+Redis transactions queue commands and execute them all at once:
+
+``` ruby
+require "async/redis"
+
+endpoint = Async::Redis.local_endpoint
+client = Async::Redis::Client.new(endpoint)
+
+Async do
+	begin
+		# Execute commands atomically:
+		results = client.transaction do |context|
+			context.multi
+			
+			# Queue commands for atomic execution:
+			context.set("user:1:name", "Alice")
+			context.set("user:1:email", "alice@example.com")
+			context.incr("user:count")
+			
+			# Execute all queued commands:
+			context.execute
+		end
+		
+		puts "Transaction results: #{results}"
+		
+	ensure
+		client.close
+	end
+end
+```
+
+### Watch/Unwatch for Optimistic Locking
+
+When multiple clients might modify the same data simultaneously, you need to handle race conditions. Redis WATCH provides optimistic locking - the transaction only executes if watched keys haven't changed since you started watching them.
+
+This is essential for scenarios like:
+- Updating counters or balances where the current value matters.
+- Implementing atomic increment operations with business logic.
+- Preventing lost updates in concurrent environments.
+
+Here's how to implement safe concurrent updates:
+
+``` ruby
+require "async/redis"
+
+endpoint = Async::Redis.local_endpoint
+client = Async::Redis::Client.new(endpoint)
+
+Async do
+	begin
+		# Initialize counter.
+		client.set("counter", 0)
+		
+		# Optimistic locking example:
+		5.times do |i|
+			success = false
+			attempts = 0
+			
+			while !success && attempts < 3
+				attempts += 1
+				
+				result = client.transaction do |context|
+					# Watch the counter for changes (executes immediately):
+					context.watch("counter")
+					
+					# Read current value (executes immediately):
+					current_value = client.get("counter").to_i
+					new_value = current_value + 1
+					
+					# Start transaction - commands after this are queued, not executed:
+					context.multi
+					
+					# Queue commands (these return "QUEUED", don't execute yet):
+					context.set("counter", new_value)
+					context.set("last_update", Time.now.to_f)
+					
+					# Execute all queued commands atomically.
+					# Returns nil if watched key was modified by another client:
+					context.execute
+				end
+				
+				if result
+					puts "Increment #{i + 1} succeeded: #{result}"
+					success = true
+				else
+					puts "Increment #{i + 1} failed, retrying (attempt #{attempts})"
+					sleep 0.01
+				end
+			end
+		end
+		
+		final_value = client.get("counter")
+		puts "Final counter value: #{final_value}"
+		
+	ensure
+		client.close
+	end
+end
+```
+
+## Pipelines
+
+When you need to execute many Redis commands quickly, sending them one-by-one creates network latency bottlenecks. Pipelines solve this by batching multiple commands together, dramatically reducing round-trip time.
+
+Use pipelines when you need:
+- **Better performance**: Reduce network round trips for bulk operations.
+- **High throughput**: Process hundreds or thousands of commands efficiently.
+- **Independent operations**: Commands that don't depend on each other's results.
+
+Unlike transactions, pipeline commands are not atomic - some may succeed while others fail:
+
+``` ruby
+require "async/redis"
+
+endpoint = Async::Redis.local_endpoint
+client = Async::Redis::Client.new(endpoint)
+
+Async do
+	begin
+		# Execute commands in a pipeline:
+		results = client.pipeline do |context|
+			# Commands are buffered and flushed if needed:
+			context.set("pipeline:1", "value1")
+			context.set("pipeline:2", "value2")
+			context.set("pipeline:3", "value3")
+			
+			context.get("pipeline:1")
+			context.get("pipeline:2")
+			context.get("pipeline:3")
+			
+			# Flush and collect the results from all previous commands:
+			context.collect
+		end
+		
+		puts "Pipeline results: #{results}"
+		
+	ensure
+		client.close
+	end
+end
+```
+
+### Synchronous Pipeline Operations
+
+When you need immediate results from individual commands within a pipeline, use `context.sync`:
+
+``` ruby
+require "async/redis"
+
+endpoint = Async::Redis.local_endpoint
+client = Async::Redis::Client.new(endpoint)
+
+Async do
+	begin
+		client.pipeline do |context|
+			# Set values using pipeline - no immediate response:
+			context.set("async_key_1", "value1")
+			context.set("async_key_2", "value2")
+			
+			# Get immediate response using sync:
+			immediate_result = context.sync.get("async_key_1")
+			puts "Immediate result: #{immediate_result}"
+			
+			# Continue with pipelined operations:
+			context.get("async_key_2")
+			
+			# Collect remaining pipelined results:
+			context.collect
+		end
+		
+	ensure
+		client.close
+	end
+end
+```
+
+Use `context.sync` when you need to:
+- **Check values mid-pipeline**: Verify data before continuing with more operations.
+- **Conditional logic**: Make decisions based on current Redis state.
+- **Debugging**: Get immediate feedback during pipeline development.
+
+Note that `sync` operations execute immediately and flush pending responses, so use them strategically to maintain pipeline performance benefits.

data/lib/async/redis/client.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 # Released under the MIT License.
-# Copyright, 2018-2024, by Samuel Williams.
+# Copyright, 2018-2025, by Samuel Williams.
 # Copyright, 2018, by Huba Nagy.
 # Copyright, 2019, by Mikael Henriksson.
 # Copyright, 2019, by David Ortiz.
@@ -9,7 +9,7 @@
 
 require_relative "context/pipeline"
 require_relative "context/transaction"
-require_relative "context/subscribe"
+require_relative "context/subscription"
 require_relative "endpoint"
 
 require "io/endpoint/host_endpoint"
@@ -32,11 +32,49 @@ module Async
 				# Subscribe to one or more channels for pub/sub messaging.
 				# @parameter channels [Array(String)] The channels to subscribe to.
 				# @yields {|context| ...} If a block is given, it will be executed within the subscription context.
-				# 	@parameter context [Context::Subscribe] The subscription context.
+				# 	@parameter context [Context::Subscription] The subscription context.
 				# @returns [Object] The result of the block if block given.
-				# @returns [Context::Subscribe] The subscription context if no block given.
+				# @returns [Context::Subscription] The subscription context if no block given.
 				def subscribe(*channels)
-					context = Context::Subscribe.new(@pool, channels)
+					context = Context::Subscription.new(@pool, channels)
+					
+					return context unless block_given?
+					
+					begin
+						yield context
+					ensure
+						context.close
+					end
+				end
+				
+				# Subscribe to one or more channel patterns for pub/sub messaging.
+				# @parameter patterns [Array(String)] The channel patterns to subscribe to.
+				# @yields {|context| ...} If a block is given, it will be executed within the subscription context.
+				# 	@parameter context [Context::Subscription] The subscription context.
+				# @returns [Object] The result of the block if block given.
+				# @returns [Context::Subscription] The subscription context if no block given.
+				def psubscribe(*patterns)
+					context = Context::Subscription.new(@pool, [])
+					context.psubscribe(patterns)
+					
+					return context unless block_given?
+					
+					begin
+						yield context
+					ensure
+						context.close
+					end
+				end
+				
+				# Subscribe to one or more sharded channels for pub/sub messaging (Redis 7.0+).
+				# @parameter channels [Array(String)] The sharded channels to subscribe to.
+				# @yields {|context| ...} If a block is given, it will be executed within the subscription context.
+				# 	@parameter context [Context::Subscription] The subscription context.
+				# @returns [Object] The result of the block if block given.
+				# @returns [Context::Subscription] The subscription context if no block given.
+				def ssubscribe(*channels)
+					context = Context::Subscription.new(@pool, [])
+					context.ssubscribe(channels)
 					
 					return context unless block_given?
 					

data/lib/async/redis/cluster_client.rb

@@ -5,12 +5,18 @@
 # Copyright, 2025, by Travis Bell.
 
 require_relative "client"
+require_relative "cluster_subscription"
+require_relative "range_map"
 require "io/stream"
 
+require "protocol/redis/cluster/methods"
+
 module Async
 	module Redis
 		# A Redis cluster client that manages multiple Redis instances and handles cluster operations.
 		class ClusterClient
+			include ::Protocol::Redis::Cluster::Methods
+			
 			# Raised when cluster configuration cannot be reloaded.
 			class ReloadError < StandardError
 			end
@@ -21,54 +27,6 @@ module Async
 			
 			Node = Struct.new(:id, :endpoint, :role, :health, :client)
 			
-			# A map that stores ranges and their associated values for efficient lookup.
-			class RangeMap
-				# Initialize a new RangeMap.
-				def initialize
-					@ranges = []
-				end
-				
-				# Add a range-value pair to the map.
-				# @parameter range [Range] The range to map.
-				# @parameter value [Object] The value to associate with the range.
-				# @returns [Object] The added value.
-				def add(range, value)
-					@ranges << [range, value]
-					
-					return value
-				end
-				
-				# Find the value associated with a key within any range.
-				# @parameter key [Object] The key to find.
-				# @yields {...} Block called if no range contains the key.
-				# @returns [Object] The value if found, result of block if given, or nil.
-				def find(key)
-					@ranges.each do |range, value|
-						return value if range.include?(key)
-					end
-					
-					if block_given?
-						return yield
-					end
-					
-					return nil
-				end
-				
-				# Iterate over all values in the map.
-				# @yields {|value| ...} Block called for each value.
-				# 	@parameter value [Object] The value from the range-value pair.
-				def each
-					@ranges.each do |range, value|
-						yield value
-					end
-				end
-				
-				# Clear all ranges from the map.
-				def clear
-					@ranges.clear
-				end
-			end
-			
 			# Create a new instance of the cluster client.
 			#
 			# @property endpoints [Array(Endpoint)] The list of cluster endpoints.
@@ -127,13 +85,35 @@ module Async
 				end
 			end
 			
+			# Get any available client from the cluster.
+			# This is useful for operations that don't require slot-specific routing, such as global pub/sub operations, INFO commands, or other cluster-wide operations.
+			# @parameter role [Symbol] The role of node to get (:master or :slave).
+			# @returns [Client] A Redis client for any available node.
+			def any_client(role = :master)
+				unless @shards
+					reload_cluster!
+				end
+				
+				# Sample a random shard to get better load distribution
+				if nodes = @shards.sample
+					nodes = nodes.select{|node| node.role == role}
+					
+					if node = nodes.sample
+						return (node.client ||= Client.new(node.endpoint, **@options))
+					end
+				end
+				
+				# Fallback to slot 0 if sampling fails
+				client_for(0, role)
+			end
+			
 			protected
 			
 			def reload_cluster!(endpoints = @endpoints)
 				@endpoints.each do |endpoint|
 					client = Client.new(endpoint, **@options)
 					
-					shards = RangeMap.new
+					shards = Async::Redis::RangeMap.new
 					endpoints = []
 					
 					client.call("CLUSTER", "SHARDS").each do |shard|
@@ -243,6 +223,32 @@ module Async
 				
 				return slots
 			end
+			
+			# Subscribe to one or more sharded channels for pub/sub messaging in cluster environment.
+			# The subscription will be created on the appropriate nodes responsible for each channel's hash slot.
+			#
+			# @parameter channels [Array(String)] The sharded channels to subscribe to.
+			# @yields {|context| ...} If a block is given, it will be executed within the subscription context.
+			# 	@parameter context [ClusterSubscription] The cluster subscription context.
+			# @returns [Object] The result of the block if block given.
+			# @returns [ClusterSubscription] The cluster subscription context if no block given.
+			def subscribe(*channels)
+				context = ClusterSubscription.new(self)
+				
+				if channels.any?
+					context.subscribe(channels)
+				end
+				
+				if block_given?
+					begin
+						yield context
+					ensure
+						context.close
+					end
+				else
+					return context
+				end
+			end
 		end
 	end
 end