async-limiter 1.5.4 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 69bd1687c9db3ae2672a5b2fb4458d127eb3521d5fc1db244f4ec7c773f32b39
-  data.tar.gz: 88b8a481dc2ad96e0b49aed1e80ddaa5871b0f8e36cd06526bf9854b32a1c23c
+  metadata.gz: 324fe64138f3bd6854bc8d5349b8404a01d6bd11ee64f5b0ad204d4154efa15b
+  data.tar.gz: 4cc2963430befada104733f236d4e59b900750c1e3eeeb73fc18f2dece26c5b2
 SHA512:
-  metadata.gz: 4a656389da97b0990f90efa4dfbef3e58d01183969018a60b77b85704c014342d55eec954d00b3cf1870ccda8430843036ffcd49bc8449dba82a9b1c4f09d240
-  data.tar.gz: e63c0d3882fe2c3ad08648295bfe58a861f3ebaf1e0321ab2e6685dab5ec1a25e4184773206278497b2b94e15f67223f268dfdc16e1fb69f8a3db3b99919105c
+  metadata.gz: 815670fb97ad80d90e9a5a6ec9034f51242fe793784220af2b3ce769b3afd49389a03a0d53ac042aa98638affef8636607976f6908f834e667ab31ac96fbb628
+  data.tar.gz: 5f8df98d26156a041023d05e9c758a17eeefca437cd9ffa34c9a102df820a3e45380dbda8e09b23dd14e189014a49554745349ff13de70647d59a7a5abe33d4d

data/context/generic-limiter.md

@@ -0,0 +1,167 @@
+# Generic Limiter
+
+This guide explains the {ruby Async::Limiter::Generic} class, which provides unlimited concurrency by default and serves as the base implementation for all other limiters. It's ideal when you need timing constraints without concurrency limits, or when building custom limiter implementations.
+
+## Usage
+
+The simplest case - no limits on concurrent execution:
+
+```ruby
+require "async"
+require "async/limiter"
+
+Async do
+	limiter = Async::Limiter::Generic.new
+	
+	# All 100 tasks run concurrently:
+	100.times do |i|
+		limiter.async do |task|
+			puts "Task #{i} running"
+			task.sleep 1
+		end
+	end
+end
+```
+
+All tasks start immediately and run in parallel, limited only by system resources.
+
+### Async Execution
+
+The primary way to use Generic limiter is through the `async` method:
+
+```ruby
+require "async"
+require "async/limiter"
+
+Async do
+	limiter = Async::Limiter::Generic.new
+
+	# Create async tasks through the limiter:
+	tasks = 5.times.map do |i|
+		limiter.async do |task|
+			puts "Task #{i} started at #{Time.now}"
+			task.sleep 1
+			puts "Task #{i} completed at #{Time.now}"
+			"result_#{i}"
+		end
+	end
+	
+	# Wait for all tasks to complete:
+	results = tasks.map(&:wait)
+	puts "All results: #{results}"
+end
+```
+
+### Sync Execution
+
+For synchronous execution within an async context:
+
+```ruby
+Async do
+	limiter = Async::Limiter::Generic.new
+	
+	# Execute synchronously within the limiter:
+	result = limiter.sync do |task|
+		puts "Executing in task: #{task}"
+		"sync result"
+	end
+	
+	puts result  # => "sync result"
+end
+```
+
+## Timing Coordination
+
+Generic limiters excel when combined with timing strategies for pure rate limiting:
+
+### Rate Limiting Without Concurrency Limits
+
+```ruby
+Async do
+	# Allow unlimited concurrency but rate limit to 10 operations per second:
+	timing = Async::Limiter::Timing::LeakyBucket.new(10.0, 50.0)
+	limiter = Async::Limiter::Generic.new(timing: timing)
+
+	# All tasks start immediately, but timing strategy controls rate:
+	100.times do |i|
+		limiter.async do |task|
+			puts "Task #{i} executing at #{Time.now}"
+			# Timing strategy ensures rate limiting.
+		end
+	end
+end
+```
+
+### Burst Handling
+
+```ruby
+Async do
+	# Allow bursts up to 20 operations, then limit to 5 per second:
+	timing = Async::Limiter::Timing::SlidingWindow.new(
+		# 1-second window:
+		1.0,
+		# Allow bursting:
+		Async::Limiter::Timing::Burst::Greedy,
+		# 5 operations per second:
+		5
+	)
+	
+	limiter = Async::Limiter::Generic.new(timing: timing)
+	
+	# First 20 operations execute immediately (burst).
+	# Subsequent operations are rate limited:
+	50.times do |i|
+		limiter.async do |task|
+			puts "Operation #{i} at #{Time.now}"
+		end
+	end
+end
+```
+
+## Advanced Usage Patterns
+
+### Cost-Based Operations
+
+When using timing strategies, you can specify different costs for operations:
+
+```ruby
+# Create limiter with timing strategy that supports costs:
+timing = Async::Limiter::Timing::LeakyBucket.new(10.0, 50.0)  # 10/sec rate, 50 capacity.
+limiter = Async::Limiter::Generic.new(timing: timing)
+
+Async do
+	# Light operations:
+	limiter.acquire(cost: 0.5) do |resource|
+		puts "Light operation using #{resource}"
+	end
+	
+	# Standard operations (default cost: 1.0):
+	limiter.acquire do |resource|
+		puts "Standard operation using #{resource}"
+	end
+	
+	# Heavy operations:
+	limiter.acquire(cost: 5.0) do |resource|
+		puts "Heavy operation using #{resource}"
+	end
+	
+	# Operations that exceed timing capacity will fail:
+	begin
+		limiter.acquire(cost: 100.0)  # Exceeds capacity of 50.0.
+	rescue ArgumentError => error
+		Console.error(self, error)
+	end
+end
+```
+
+Note that by default, lower cost operations will occur before higher cost operations. In other words, low cost operations will starve out higher cost operations unless you use {ruby Async::Limiter::Timing::Ordered} to force FIFO acquires.
+
+```ruby
+# Default behavior - potential starvation:
+timing = Async::Limiter::Timing::LeakyBucket.new(2.0, 10.0)
+
+# FIFO ordering - prevents starvation:
+timing = Async::Limiter::Timing::Ordered.new(
+	Async::Limiter::Timing::LeakyBucket.new(2.0, 10.0)
+)
+```

data/context/getting-started.md

@@ -0,0 +1,226 @@
+# Getting Started
+
+This guide explains how to get started the `async-limiter` gem for controlling concurrency and rate limiting in Ruby applications.
+
+## Installation
+
+Add the gem to your project:
+
+```bash
+$ bundle add async-limiter
+```
+
+## Core Concepts
+
+`async-limiter` provides three main limiter classes that can be combined with timing strategies:
+
+### Limiter Classes
+
+- **{ruby Async::Limiter::Generic}** - Unlimited concurrency (default behavior).
+- **{ruby Async::Limiter::Limited}** - Enforces a concurrency limit (counting semaphore).
+- **{ruby Async::Limiter::Queued}** - Queue-based limiter with priority/timeout support.
+
+### Timing Strategies
+
+- **{ruby Async::Limiter::Timing::None}** - No timing constraints (default).
+- **{ruby Async::Limiter::Timing::SlidingWindow}** - Continuous rolling time windows.
+- **{ruby Async::Limiter::Timing::FixedWindow}** - Discrete time boundaries.
+- **{ruby Async::Limiter::Timing::LeakyBucket}** - Token bucket with automatic leaking.
+
+## Usage
+
+The simplest case - no limits:
+
+```ruby
+require "async"
+require "async/limiter"
+
+Async do
+	limiter = Async::Limiter::Generic.new
+	
+	# All tasks run concurrently:
+	100.times do |i|
+		limiter.async do |task|
+			puts "Task #{i} running"
+			task.sleep 1
+		end
+	end
+end
+```
+
+### Concurrency Limiting
+
+Limit the number of concurrent tasks:
+
+```ruby
+require "async"
+require "async/limiter"
+
+Async do
+	# Max 2 concurrent tasks:
+	limiter = Async::Limiter::Limited.new(2)
+	
+	4.times do |i|
+		limiter.async do |task|
+			puts "Task #{i} started"
+			task.sleep 1
+			puts "Task #{i} finished"
+		end
+	end
+end
+```
+
+This runs a maximum of 2 tasks concurrently. Total duration is 2 seconds (tasks 0,1 run first, then tasks 2,3).
+
+### Timeouts
+
+You can control how long to wait when acquiring resources using the `timeout` parameter. This is particularly useful when working with limited capacity limiters that might block indefinitely.
+
+```ruby
+require "async"
+require "async/limiter"
+
+Async do
+	# Zero limit will always block:
+	limiter = Async::Limiter::Limited.new(0)
+	
+	limiter.acquire(timeout: 3)
+	# => nil
+
+	limiter.acquire(timeout: 3) do
+		puts "Acquired."
+	end or puts "Timed out!"
+end
+```
+
+**Key timeout behaviors:**
+
+- `timeout: nil` (default) - Wait indefinitely until a resource becomes available
+- `timeout: 0` - Non-blocking operation; return immediately if no resource is available  
+- `timeout: N` (where N > 0) - Wait up to N seconds for a resource to become available
+
+**Return values:**
+- Returns `true` (or the acquired resource) when successful
+- Returns `nil` when the timeout is exceeded or no resource is available
+
+## Rate Limiting
+
+Timing strategies can be used to implement rate limiting, for example a continuous rolling time windows:
+
+```ruby
+require "async"
+require "async/limiter"
+
+Async do
+	# Max 3 tasks within any 1-second sliding window
+	timing = Async::Limiter::Timing::SlidingWindow.new(
+		1.0, # 1-second window.
+		Async::Limiter::Timing::Burst::Greedy, # Allow bursting
+		3 # 3 tasks per window
+	)
+	
+	limiter = Async::Limiter::Limited.new(10, timing: timing)
+	
+	10.times do |i|
+		limiter.async do |task|
+			puts "Task #{i} started at #{Time.now}"
+			task.sleep 0.5
+		end
+	end
+end
+```
+
+### Variable Cost Operations
+
+Rate limiting by default works with unit costs - each acquire consumes 1 unit of capacity. However, in more complex situations, you may want to use variable costs to model different operation weights:
+
+```ruby
+require "async"
+require "async/limiter"
+
+Async do
+	# Leaky bucket: 2 tokens/second, capacity 10
+	timing = Async::Limiter::Timing::LeakyBucket.new(2.0, 10.0)
+	limiter = Async::Limiter::Limited.new(100, timing: timing)
+	
+	# Light operations consume fewer tokens:
+	limiter.acquire(cost: 0.5) do
+		puts "Light database query"
+	end
+	
+	# Heavy operations consume more tokens:
+	limiter.acquire(cost: 5.0) do
+		puts "Complex ML inference"
+	end
+end
+```
+
+**Cost represents the resource weight** of each operation:
+- `cost: 0.5` - Light operations (quick queries, cache reads).
+- `cost: 1.0` - Standard operations (default).
+- `cost: 5.0` - Heavy operations (complex computations, large uploads).
+
+#### Starvation and Head-of-Line Blocking
+
+**Variable costs introduce two important fairness issues:**
+
+**1. Starvation Problem:**
+High-cost operations can be indefinitely delayed by streams of low-cost operations:
+
+```ruby
+# Without ordering - starvation can occur
+timing = Async::Limiter::Timing::LeakyBucket.new(2.0, 10.0)
+limiter = Async::Limiter::Limited.new(100, timing: timing)
+
+# High-cost task starts waiting for 8.0 tokens
+limiter.acquire(cost: 8.0) do
+	puts "Expensive operation"  # May never execute!
+end
+
+# Continuous stream of small operations consume tokens as they become available
+100.times do |i|
+	limiter.acquire(cost: 0.5) do
+		puts "Quick operation #{i}"  # These keep running
+	end
+end
+```
+
+**2. Head-of-Line Blocking:**
+When using FIFO ordering to prevent starvation, large operations can block smaller ones:
+
+```ruby
+# With ordering - prevents starvation but creates head-of-line blocking
+ordered_timing = Async::Limiter::Timing::Ordered.new(timing)
+fair_limiter = Async::Limiter::Limited.new(100, timing: ordered_timing)
+
+# Large operation blocks the queue
+fair_limiter.acquire(cost: 8.0) do
+	puts "Expensive operation (takes time to get tokens)"
+end
+
+# These must wait even though they need fewer tokens
+fair_limiter.acquire(cost: 0.5) { puts "Quick op 1" }  # Blocked
+fair_limiter.acquire(cost: 0.5) { puts "Quick op 2" }  # Blocked
+```
+
+#### Choosing the Right Strategy
+
+**Use Unordered (default) when:**
+- Maximum throughput is critical
+- Operations have similar costs
+- Occasional starvation is acceptable
+
+**Use Ordered when:**
+- Fairness is more important than efficiency
+- Starvation would be unacceptable
+- Predictable execution order is required
+
+```ruby
+# Unordered: Higher throughput, possible starvation
+timing = Async::Limiter::Timing::LeakyBucket.new(2.0, 10.0)
+
+# Ordered: Fair execution, lower throughput
+ordered_timing = Async::Limiter::Timing::Ordered.new(timing)
+```
+
+The choice depends on whether your application prioritizes **efficiency** (unordered) or **fairness** (ordered).

data/context/index.yaml

@@ -0,0 +1,41 @@
+# Automatically generated context index for Utopia::Project guides.
+# Do not edit then files in this directory directly, instead edit the guides and then run `bake utopia:project:agent:context:update`.
+---
+description: Execution rate limiting for Async
+metadata:
+  documentation_uri: https://socketry.github.io/async-limiter/
+  source_code_uri: https://github.com/socketry/async-limiter.git
+files:
+- path: getting-started.md
+  title: Getting Started
+  description: This guide explains how to get started the `async-limiter` gem for
+    controlling concurrency and rate limiting in Ruby applications.
+- path: generic-limiter.md
+  title: Generic Limiter
+  description: This guide explains the <code class="language-ruby">Async::Limiter::Generic</code>
+    class, which provides unlimited concurrency by default and serves as the base
+    implementation for all other limiters. It's ideal when you need timing constraints
+    without concurrency limits, or when building custom limiter implementations.
+- path: limited-limiter.md
+  title: Limited Limiter
+  description: This guide explains the <code class="language-ruby">Async::Limiter::Limited</code>
+    class, which provides semaphore-style concurrency control, enforcing a maximum
+    number of concurrent operations. It's perfect for controlling concurrency when
+    you have limited capacity or want to prevent system overload.
+- path: queued-limiter.md
+  title: Queued Limiter
+  description: This guide explains the <code class="language-ruby">Async::Limiter::Queued</code>
+    class, which provides priority-based task scheduling with optional resource management.
+    Its key feature is priority-based acquisition where higher priority tasks get
+    access first, with optional support for distributing specific resources from a
+    pre-populated queue.
+- path: timing-strategies.md
+  title: Timing Strategies
+  description: This guide explains how to use timing strategies to provide rate limiting
+    and timing constraints that can be combined with any limiter. They control *when*
+    operations can execute, while limiters control *how many* can execute concurrently.
+- path: token-usage.md
+  title: Token Usage
+  description: This guide explains how to use tokens for advanced resource management
+    with `async-limiter`. Tokens provide sophisticated resource handling with support
+    for re-acquisition and automatic cleanup.

data/context/limited-limiter.md

@@ -0,0 +1,184 @@
+# Limited Limiter
+
+This guide explains the {ruby Async::Limiter::Limited} class, which provides semaphore-style concurrency control, enforcing a maximum number of concurrent operations. It's perfect for controlling concurrency when you have limited capacity or want to prevent system overload.
+
+## Usage
+
+Limit the number of concurrent tasks:
+
+```ruby
+require "async"
+require "async/limiter"
+
+Async do
+	# Maximum 2 concurrent tasks
+	limiter = Async::Limiter::Limited.new(2)
+	
+	4.times do |i|
+		limiter.async do |task|
+			puts "Task #{i} started at #{Time.now}"
+			task.sleep 1
+			puts "Task #{i} finished at #{Time.now}"
+		end
+	end
+end
+
+# Output shows tasks 0,1 run first, then tasks 2,3
+# Total duration: ~2 seconds instead of ~1 second
+```
+
+### Block-Based Acquisition
+
+The recommended pattern using automatic cleanup:
+
+```ruby
+limiter = Async::Limiter::Limited.new(1)
+
+# Acquire with automatic release using blocks:
+limiter.acquire do |acquired|
+	puts "I have acquired: #{acquired}"
+	# Automatically released when block exits.
+end
+```
+
+## Timeouts
+
+All acquisition methods support comprehensive timeout options:
+
+```ruby
+limiter = Async::Limiter::Limited.new(1)
+
+Async do
+	# Non-blocking (immediate check) - should succeed:
+	if limiter.acquire(timeout: 0)
+		puts "Got acquisition immediately"
+	else
+		puts "No capacity available"
+	end
+	
+	# Now limiter is at capacity, so subsequent calls will fail/timeout.
+	
+	# Non-blocking check - will fail since capacity is used:
+	if limiter.acquire(timeout: 0)
+		puts "Got second acquisition"
+	else
+		puts "No capacity available for second acquisition"
+	end
+	
+	# Timed acquisition - will timeout since capacity is still used:
+	if limiter.acquire(timeout: 0.1)
+		puts "Got acquisition within timeout"
+	else
+		puts "Timed out waiting for capacity"
+	end
+	
+	# With blocks (automatic cleanup):
+	result = limiter.acquire(timeout: 1.0) do |acquired|
+		"Successfully acquired and used"
+	end
+	
+	puts result || "Acquisition timed out"
+end
+```
+
+### Concurrent Timeout Behavior
+
+The limiter prevents convoy effects where quick timeouts aren't blocked by slow ones:
+
+```ruby
+limiter = Async::Limiter::Limited.new(1)
+Async do
+	limiter.acquire  # Fill to capacity.
+
+	results = []
+
+	# Start multiple tasks with different timeouts:
+	tasks = [
+		Async {limiter.acquire(timeout: 1.0); results << "Long timeout."},
+		Async {limiter.acquire(timeout: 0.1); results << "Short timeout."},
+		Async {limiter.acquire(timeout: 0);   results << "Non-blocking."},
+	]
+
+	# All tasks complete quickly, even with a long timeout task present:
+	tasks.map(&:wait)
+	puts results
+	# => ["Non-blocking.", "Short timeout.", "Long timeout."]
+end
+```
+
+## Dynamic Limit Adjustment
+
+Adjust limits at runtime based on changing conditions:
+
+```ruby
+limiter = Async::Limiter::Limited.new(2)
+puts "Initial limit: #{limiter.limit}"  # 2
+
+# Increase capacity during high load
+limiter.limit = 5
+puts "Increased limit: #{limiter.limit}"  # 5
+
+# Decrease capacity during high load
+limiter.limit = 1
+puts "Decreased limit: #{limiter.limit}"  # 1
+```
+
+## Cost-Based Operations
+
+Operations can consume multiple "units" based on their computational weight:
+
+```ruby
+# Create limiter with timing strategy that has capacity limits:
+timing = Async::Limiter::Timing::LeakyBucket.new(5.0, 10.0)  # 5/sec rate, 10 capacity.
+limiter = Async::Limiter::Limited.new(100, timing: timing)
+
+Async do
+	# Light operations (consume 0.5 units):
+	limiter.acquire(cost: 0.5) do
+		perform_light_database_query()
+	end
+	
+	# Normal operations (default cost: 1.0):
+	limiter.acquire do
+		perform_standard_operation()
+	end
+	
+	# Heavy operations (consume 3.5 units):
+	limiter.acquire(cost: 3.5) do
+		perform_heavy_computation()
+	end
+	
+	# Operations exceeding capacity fail fast:
+	begin
+		# Exceeds timing capacity of 10.0:
+		limiter.acquire(cost: 15.0)
+	rescue ArgumentError => error
+		puts "#{error.message}"
+		# => Cost 15.0 exceeds maximum supported cost 10.0
+	end
+end
+```
+
+### Cost + Timeout Combinations
+
+When using cost-based operations with timing strategies, be aware that high-cost operations can be starved by continuous low-cost operations. Use {ruby Async::Limiter::Timing::Ordered} to enforce FIFO ordering if fairness is important:
+
+```ruby
+# Default behavior - potential starvation:
+timing = Async::Limiter::Timing::LeakyBucket.new(2.0, 10.0)
+limiter = Async::Limiter::Limited.new(100, timing: timing)
+
+# High-cost operation might be starved by many small operations:
+result = limiter.acquire(timeout: 30.0, cost: 8.0) do |acquired|
+	expensive_machine_learning_inference()
+end
+
+# With FIFO ordering - prevents starvation:
+ordered_timing = Async::Limiter::Timing::Ordered.new(timing)
+fair_limiter = Async::Limiter::Limited.new(100, timing: ordered_timing)
+
+# High-cost operation is guaranteed to execute in arrival order:
+result = fair_limiter.acquire(timeout: 30.0, cost: 8.0) do |acquired|
+	expensive_machine_learning_inference()
+end
+```