counting_semaphore 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cffd9d409923265971a2d4b2994a22b5a0a84ce57552f90ef3b0ce4d3ddb0c3d
-  data.tar.gz: d8ffaf1fa6d13ece63fed8d75bcd687d00450f378e93c19f06a4669f8722a457
+  metadata.gz: f8aa18aa9fc4f48207c718855f024c62dc6034f8708cfb9ce6adf530502fddb2
+  data.tar.gz: 108b3734991093af91e790c347f3ae83a3ca187bfb647648745a7c7cc8d89f7d
 SHA512:
-  metadata.gz: 3c59f43dfdac58fbe4a2c660138a7f6a2d2a70ead0afa85d5acf666fd72018a064a814cf933251ea4f21858ffd6e94931a5baacf4fdca04384d2ff8c76149175
-  data.tar.gz: ffb43973632962da0194d463688e82d7125fcafef9e1546e9a89a180c5ed22670cbf4d10061fb13df5586342f4e81a7b2324c6631074aaa4e6d9f56f65103b82
+  metadata.gz: 43bf0f8f4ff7222f87d934ca2a26d154a33b44bd9639f9c8c1868322c0e6e92103a95f85e05ddc5f2756768ab6c8435fc82aebdf7ca6eaed8554a987693eb3f2
+  data.tar.gz: d365b7adf32b5462454d69f3358539bb8c16c8efbca2c9ec70abd51c4a7e302ec976615b296d4656598062cd9e4850f30bb935325f9db5ebcf8295a247aae7b7

data/README.md

@@ -9,7 +9,7 @@ A counting semaphore implementation for Ruby with local and distributed (Redis)
 
 ## What is it for?
 
-When you have a metered and limited resource that only supports a certain number of simultaneous operations you need a [semaphore](https://en.wikipedia.org/wiki/Semaphore_(programming)) primitive. In Ruby, a semaphore usually controls access to "one whole resource":
+When you have a _metered and limited_ resource that only supports a certain number of simultaneous operations you need a [semaphore](https://en.wikipedia.org/wiki/Semaphore_(programming)) primitive. In Ruby, most semaphores usually controls access "one whole resource":
 
 ```ruby
 sem = Semaphore.new
@@ -18,45 +18,191 @@ sem.with_lease do
 end
 ```
 
-This is well covered - for example - by POSIX semaphores if you are within one machine, or by the venerable [redis-semaphore](https://github.com/dv/redis-semaphore)
+This is well covered - for example - by POSIX semaphores if you are within one machine, and is known as a _binary semaphore_ (it is either "open" or "closed"). There are also _counting_ semaphores where you permit N of leases to be taken, which is available in the venerable [redis-semaphore](https://github.com/dv/redis-semaphore) gem.
 
 The problem comes if you need to hold access to a certain _amount_ of a resource. For example, you know that you are doing 5 expensive operations in bulk, and you know that your entire application can only be doing 20 in total - governed by the API access limits. For that, you need a [counting semaphore](https://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Semaphore.html#acquire-instance_method) - such a semaphore is provided by [concurrent-ruby](https://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Semaphore.html#acquire-instance_method) for example. It allows you to acquire a certain number of _permits_ and then release them.
 
-This library does the same and also has a simple `LocalSemaphore` which can be used across threads or fibers. This allows for coordination if you are only running one process/Ractor. It is thread-safe and fairly simple in operation:
+This library provides both a simple `LocalSemaphore` which can be used across threads or fibers, and a Redis-based `RedisSemaphore` for coordination across processes and machines. Both implement a Lease-based API compatible with concurrent-ruby's Semaphore.
+
+## Usage
+
+### Basic Usage with `with_lease`
+
+The recommended way to use the semaphore is with the `with_lease` method, which provides automatic cleanup:
 
 ```ruby
-require "counting_semaphore"
+require 'counting_semaphore'
 
-# Create a semaphore that allows up to 10 concurrent operations
+# Create a local semaphore with capacity of 10
 semaphore = CountingSemaphore::LocalSemaphore.new(10)
 
-# Do an operation that occupies 2 slots
-semaphore.with_lease(2) do
-  # This block can only run when 2 tokens are available
-  # Do your work here
-  puts "Doing work that requires 2 tokens"
+# Acquire 3 permits and automatically release on block exit
+semaphore.with_lease(3, timeout_seconds: 10) do
+  puts "Holding 3 permits"
+  # Do your work here - permits are automatically released when the block exits
 end
 ```
 
-However, we also include a Redis based _shared counting semaphore_ which you can use for resource access control across processes and across machines - provided they have access to a shared Redis server. The semaphore is identified by a _namespace_ - think of it as the `id` of the semaphore. Leases are obtained and released using Redis blocking operations and Lua scripts (for atomicity):
+The block receives the lease object, which you can inspect:
 
 ```ruby
-require "counting_semaphore"
-require "redis"  # Redis is required for RedisSemaphore
+semaphore.with_lease(3) do |lease|
+  puts "Holding #{lease.permits} permits (ID: #{lease.id})"
+  # Automatic cleanup on block exit
+end
+```
 
-# Create a Redis semaphore using Redis
-# You can also pass your ConnectionPool instance.
-redis = Redis.new
-semaphore = CountingSemaphore::RedisSemaphore.new(10, "api_ratelimit", redis:)
+### Distributed Semaphore with Redis
+
+The Redis semaphore works identically but coordinates across processes and machines:
 
-# and then use it the same as the LocalSemaphore
+```ruby
+require 'redis'
+
+redis = Redis.new
+semaphore = CountingSemaphore::RedisSemaphore.new(
+  10,                    # capacity
+  "api_ratelimit",       # namespace (unique identifier)
+  redis: redis,
+  lease_ttl_seconds: 60  # lease expires after 60 seconds
+)
+
+# Use it the same way - works across multiple processes
 semaphore.with_lease(3) do
-  # This block can only run when 3 tokens are available
-  # Works across multiple processes/machines
-  puts "Doing distributed work"
+  puts "Doing distributed work with 3 permits"
+  # Permits automatically released when done
+end
+```
+
+### Checking Availability
+
+You can query the current state of the semaphore:
+
+```ruby
+puts "Available permits: #{semaphore.available_permits}"
+puts "Capacity: #{semaphore.capacity}"
+puts "Currently in use: #{semaphore.currently_leased}"
+```
+
+### Advanced: Manual Lease Control
+
+For more control, you can manually acquire and release leases. This is useful when you can't use a block structure:
+
+```ruby
+# Acquire permits (returns a Lease object)
+lease = semaphore.acquire(2)
+
+begin
+  # Do some work
+  puts "Working with 2 permits..."
+ensure
+  # Always release the lease
+  semaphore.release(lease)
+end
+```
+
+#### Try Acquire with Timeout
+
+```ruby
+# Try to acquire immediately (returns nil if not available)
+lease = semaphore.try_acquire(1)
+if lease
+  begin
+    puts "Got the permit!"
+  ensure
+    semaphore.release(lease)
+  end
+else
+  puts "Could not acquire permit"
 end
+
+# Try to acquire with timeout
+lease = semaphore.try_acquire(2, 5.0)  # Wait up to 5 seconds
+if lease
+  begin
+    # Work with the permits
+  ensure
+    semaphore.release(lease)
+  end
+end
+```
+
+#### Drain All Available Permits
+
+```ruby
+# Acquire all currently available permits
+drained_lease = semaphore.drain_permits
+
+if drained_lease
+  begin
+    puts "Drained #{drained_lease.permits} permits for exclusive access"
+    # Do exclusive work
+  ensure
+    semaphore.release(drained_lease)
+  end
+end
+```
+
+### Key Benefits
+
+1. **Automatic Cleanup**: `with_lease` ensures permits are always released
+2. **Type Safety**: Lease objects ensure you can only release what you've acquired
+3. **Cross-Semaphore Protection**: Can't accidentally release a lease to the wrong semaphore
+4. **Distributed Coordination**: Redis semaphore works seamlessly across processes and machines
+5. **Lease Expiration**: Redis leases automatically expire to prevent deadlocks
+
+## Design Philosophy
+
+This library aims for compatibility with [`Concurrent::Semaphore`](https://ruby-concurrency.github.io/concurrent-ruby/1.1.5/Concurrent/Semaphore.html) from the concurrent-ruby gem, but with a key difference to support both local and distributed implementations.
+
+### How It Works
+
+The core difference from `Concurrent::Semaphore` is that **`acquire` returns a lease object** that must be passed to `release`, rather than using numeric permit counts for both operations:
+
+```ruby
+# concurrent-ruby style
+semaphore.acquire(2)
+# ... work ...
+semaphore.release(2)  # Must remember the count!
+
+# counting_semaphore style
+lease = semaphore.acquire(2)
+# ... work ...
+semaphore.release(lease)  # Lease knows its own count
 ```
 
+#### Why Not 100% API Parity?
+
+The `Concurrent::Semaphore` API where `acquire(n)` and `release(n)` use arbitrary counts works well for in-memory semaphores, but creates challenges for distributed Redis-based implementations:
+
+1. **Individual leases need TTLs**: In Redis, each lease must have an expiration to prevent deadlocks from crashed processes
+2. **Lease tracking is essential**: Distributed systems need unique identifiers for each acquired lease
+3. **Cross-process coordination**: Releasing "2 permits" doesn't map cleanly to "which 2 leases?" across processes
+4. **Ownership semantics**: The lease object makes it explicit what you acquired and what you're releasing
+
+#### The Lease Object
+
+A lease is a simple struct that contains:
+- `semaphore` - reference to the semaphore it came from
+- `id` - unique identifier (local counter for LocalSemaphore, Redis key for RedisSemaphore)
+- `permits` - number of permits held
+
+This design:
+- **Prevents bugs**: Can't accidentally release the wrong amount or to the wrong semaphore
+- **Works for both implementations**: LocalSemaphore and RedisSemaphore use the same API
+- **Follows familiar patterns**: Similar to file handles, database connections, and other resource management
+- **Maintains compatibility**: The `with_lease` block form works identically to concurrent-ruby's usage
+
+#### Query Methods
+
+The library provides the same query methods as `Concurrent::Semaphore`:
+
+- `available_permits` - returns the number of permits currently available
+- `capacity` - returns the total capacity of the semaphore
+- `currently_leased` - returns the number of permits currently in use
+
+Additionally, `drain_permits` returns a lease object (or nil) instead of an integer, maintaining consistency with the lease-based design.
+
 ## Installation
 
 Add this line to your application's Gemfile:

data/Rakefile

@@ -9,4 +9,9 @@ Rake::TestTask.new do |t|
   t.verbose = true
 end
 
-task default: [:test, :standard]
+task :generate_typedefs do
+  `bundle exec sord rbi/counting_semaphore.rbi`
+  `bundle exec sord sig/counting_semaphore.rbs`
+end
+
+task default: [:test, :standard, :generate_typedefs]

data/lib/counting_semaphore/local_semaphore.rb

@@ -1,7 +1,12 @@
+# frozen_string_literal: true
+
 # A counting semaphore that allows up to N concurrent operations.
 # When capacity is exceeded, operations block until resources become available.
+# API compatible with concurrent-ruby's Semaphore class.
 module CountingSemaphore
   class LocalSemaphore
+    include WithLeaseSupport
+
     SLEEP_WAIT_SECONDS = 0.25
 
     # @return [Integer]
@@ -9,86 +14,163 @@ module CountingSemaphore
 
     # Initialize the semaphore with a maximum capacity.
     #
-    # @param capacity [Integer] Maximum number of concurrent operations allowed
+    # @param capacity [Integer] Maximum number of concurrent operations allowed (also called permits)
     # @param logger [Logger] the logger
     # @raise [ArgumentError] if capacity is not positive
     def initialize(capacity, logger: CountingSemaphore::NullLogger)
-      raise ArgumentError, "Capacity must be positive, got #{capacity}" unless capacity > 0
+      raise ArgumentError, "Capacity must be positive, got #{capacity}" unless capacity >= 1
       @capacity = capacity.to_i
-      @leased = 0
+      @acquired = 0
       @mutex = Mutex.new
       @condition = ConditionVariable.new
       @logger = logger
     end
 
-    # Acquire a lease for the specified number of tokens and execute the block.
-    # Blocks until sufficient resources are available.
+    # Acquires the given number of permits from this semaphore, blocking until all are available.
+    #
+    # @param permits [Integer] Number of permits to acquire (default: 1)
+    # @return [CountingSemaphore::Lease] A lease object that must be passed to release()
+    # @raise [ArgumentError] if permits is not an integer or is less than one
+    def acquire(permits = 1)
+      permits = permits.to_i
+      raise ArgumentError, "Permits must be at least 1, got #{permits}" if permits < 1
+      if permits > @capacity
+        raise ArgumentError, "Cannot acquire #{permits} permits as capacity is only #{@capacity}"
+      end
+
+      loop do
+        acquired = @mutex.synchronize do
+          if (@capacity - @acquired) >= permits
+            @acquired += permits
+            @logger.debug { "Acquired #{permits} permits, now #{@acquired}/#{@capacity}" }
+            true
+          else
+            false
+          end
+        end
+
+        if acquired
+          lease_id = "local_#{object_id}_#{Time.now.to_f}_#{rand(1000000)}"
+          return CountingSemaphore::Lease.new(
+            semaphore: self,
+            id: lease_id,
+            permits: permits
+          )
+        end
+
+        @logger.debug { "Unable to acquire #{permits} permits, #{@acquired}/#{@capacity} in use, waiting" }
+        @mutex.synchronize do
+          @condition.wait(@mutex)
+        end
+      end
+    end
+
+    # Releases a previously acquired lease, returning the permits to the semaphore.
     #
-    # @param token_count [Integer] Number of tokens to acquire
-    # @param timeout_seconds [Integer] Maximum time to wait for lease acquisition (default: 30 seconds)
-    # @yield The block to execute while holding the lease
-    # @return The result of the block
-    # @raise [ArgumentError] if token_count is negative or exceeds the semaphore capacity
-    # @raise [CountingSemaphore::LeaseTimeout] if lease cannot be acquired within timeout
-    def with_lease(token_count_num = 1, timeout_seconds: 30)
-      token_count = token_count_num.to_i
-      raise ArgumentError, "Token count must be non-negative, got #{token_count}" if token_count < 0
-      if token_count > @capacity
-        raise ArgumentError, "Cannot lease #{token_count} slots as we only allow #{@capacity}"
+    # @param lease [CountingSemaphore::Lease] The lease object returned by acquire() or try_acquire()
+    # @return [nil]
+    # @raise [ArgumentError] if lease belongs to a different semaphore
+    def release(lease)
+      unless lease.semaphore == self
+        raise ArgumentError, "Lease belongs to a different semaphore"
       end
 
-      # Handle zero tokens case - no waiting needed
-      return yield if token_count.zero?
+      permits = lease.permits
 
-      did_accept = false
-      start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      @mutex.synchronize do
+        @acquired -= permits
+        @logger.debug { "Released #{permits} permits (lease: #{lease.id}), now #{@acquired}/#{@capacity}" }
+        @condition.broadcast # Signal waiting threads
+      end
+      nil
+    end
+
+    # Acquires the given number of permits from this semaphore, only if all are available
+    # at the time of invocation or within the timeout interval.
+    #
+    # @param permits [Integer] Number of permits to acquire (default: 1)
+    # @param timeout [Numeric, nil] Number of seconds to wait, or nil to return immediately (default: nil)
+    # @return [CountingSemaphore::Lease, nil] A lease object if successful, nil otherwise
+    # @raise [ArgumentError] if permits is not an integer or is less than one
+    def try_acquire(permits = 1, timeout: nil)
+      permits = permits.to_i
+      raise ArgumentError, "Permits must be at least 1, got #{permits}" if permits < 1
+      if permits > @capacity
+        return nil
+      end
+
+      start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC) if timeout
 
       loop do
         # Check timeout
-        elapsed_time = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time
-        if elapsed_time >= timeout_seconds
-          raise CountingSemaphore::LeaseTimeout.new(token_count, timeout_seconds, self)
+        if timeout
+          elapsed_time = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time
+          return nil if elapsed_time >= timeout
         end
 
-        did_accept = @mutex.synchronize do
-          if (@capacity - @leased) >= token_count
-            @leased += token_count
+        acquired = @mutex.synchronize do
+          if (@capacity - @acquired) >= permits
+            @acquired += permits
+            @logger.debug { "Acquired #{permits} permits (try), now #{@acquired}/#{@capacity}" }
             true
           else
             false
           end
         end
 
-        if did_accept
-          @logger.debug { "Leased #{token_count} and now in use #{@leased}/#{@capacity}" }
-          return yield
+        if acquired
+          lease_id = "local_#{object_id}_#{Time.now.to_f}_#{rand(1000000)}"
+          return CountingSemaphore::Lease.new(
+            semaphore: self,
+            id: lease_id,
+            permits: permits
+          )
         end
 
-        @logger.debug { "Unable to lease #{token_count}, #{@leased}/#{@capacity} waiting" }
+        # If no timeout, return immediately
+        return nil if timeout.nil?
+
+        # Wait with remaining timeout
+        remaining_timeout = timeout - (Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time)
+        return nil if remaining_timeout <= 0
 
-        # Wait on condition variable with remaining timeout
-        remaining_timeout = timeout_seconds - elapsed_time
-        if remaining_timeout > 0
-          @mutex.synchronize do
-            @condition.wait(@mutex, remaining_timeout)
-          end
-        end
-      end
-    ensure
-      if did_accept
-        @logger.debug { "Returning #{token_count} leased slots" }
         @mutex.synchronize do
-          @leased -= token_count
-          @condition.broadcast # Signal waiting threads
+          @condition.wait(@mutex, remaining_timeout)
         end
       end
     end
 
-    # Get the current number of tokens currently leased
+    # Returns the current number of permits available in this semaphore.
+    #
+    # @return [Integer] Number of available permits
+    def available_permits
+      @mutex.synchronize { @capacity - @acquired }
+    end
+
+    # Acquires and returns all permits that are immediately available.
+    # Returns a single lease representing all drained permits.
     #
-    # @return [Integer] Number of tokens currently in use
-    def currently_leased
-      @mutex.synchronize { @leased }
+    # @return [CountingSemaphore::Lease, nil] A lease for all available permits, or nil if none available
+    def drain_permits
+      permits = @mutex.synchronize do
+        available = @capacity - @acquired
+        if available > 0
+          @acquired = @capacity
+          @logger.debug { "Drained #{available} permits" }
+          available
+        else
+          0
+        end
+      end
+
+      if permits > 0
+        lease_id = "local_#{object_id}_#{Time.now.to_f}_#{rand(1000000)}"
+        CountingSemaphore::Lease.new(
+          semaphore: self,
+          id: lease_id,
+          permits: permits
+        )
+      end
     end
   end
 end

data/lib/counting_semaphore/null_logger.rb

@@ -1,28 +1,55 @@
+# frozen_string_literal: true
+
 module CountingSemaphore
-  # A null logger that discards all log messages
-  # Provides the same interface as a real logger but does nothing
+  # A null logger that discards all log messages.
+  # Provides the same interface as a real logger but does nothing.
   # Only yields blocks when ENV["RUN_ALL_LOGGER_BLOCKS"] is set to "yes",
   # which is useful in testing. Block form for Logger calls allows you
   # to skip block evaluation if the Logger level is higher than your
   # call, and thus bugs can nest in those Logger blocks. During
   # testing it is helpful to excercise those blocks unconditionally.
   module NullLogger
+    # Logs a debug message. Discards the message but may yield the block for testing.
+    #
+    # @param message [String, nil] Optional message to log (discarded)
+    # @yield Optional block that will only be executed if ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
+    # @return [nil]
     def debug(message = nil, &block)
       yield if block_given? && ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
     end
 
+    # Logs an info message. Discards the message but may yield the block for testing.
+    #
+    # @param message [String, nil] Optional message to log (discarded)
+    # @yield Optional block that will only be executed if ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
+    # @return [nil]
     def info(message = nil, &block)
       yield if block_given? && ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
     end
 
+    # Logs a warning message. Discards the message but may yield the block for testing.
+    #
+    # @param message [String, nil] Optional message to log (discarded)
+    # @yield Optional block that will only be executed if ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
+    # @return [nil]
     def warn(message = nil, &block)
       yield if block_given? && ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
     end
 
+    # Logs an error message. Discards the message but may yield the block for testing.
+    #
+    # @param message [String, nil] Optional message to log (discarded)
+    # @yield Optional block that will only be executed if ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
+    # @return [nil]
     def error(message = nil, &block)
       yield if block_given? && ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
     end
 
+    # Logs a fatal message. Discards the message but may yield the block for testing.
+    #
+    # @param message [String, nil] Optional message to log (discarded)
+    # @yield Optional block that will only be executed if ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
+    # @return [nil]
     def fatal(message = nil, &block)
       yield if block_given? && ENV["RUN_ALL_LOGGER_BLOCKS"] == "yes"
     end