async 2.32.0 → 2.39.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5a816b65e2cdf3f1f9ee99d421b32f13ed00c858bfa2b1ff0f814cf33c9dd470
-  data.tar.gz: 85dc9dfad110e4ae2f0d4db45db9b1f941b797891f945da1caaa302df233f813
+  metadata.gz: 1d8427e86e8ab22c81e41dd6e7df7bd79087e3fa72ce786f07ae7b65d9d94545
+  data.tar.gz: 30e9e7e88b211aa21afe844e3f9510b5af39d2b4af1d97710ce533f581b5c57b
 SHA512:
-  metadata.gz: e6cf114ad81e9eb511504ed44b719f9838ad84623fea14aab43acd531a9d8493b92eb0a24a49bb410670c0be82c8de5617280f39e984f1d9477c4828d96d2fc5
-  data.tar.gz: 2acb3e1516b2670a57cb717713d882f9d75807864924bf7aa691aa82cd5406a2b273c7aa813a30037a3dabce338095eb7faa5efff8f5f5b56290a07b3b0927b5
+  metadata.gz: 337694b65547afc0d4c1568a02c0c08d03d0d70a8b2319b9b9a77660ad9f37c82ff3ab49a3856ebee477657df6b254d62e59ca6b31e9312484c02e696deab007
+  data.tar.gz: e9bc3a1a27f9ba9ea6004dee40df9b385d7959c30e7059521805e91d1f64e9ed16bbc0920b4eb3b1dae8a727ab52ed8258f95bb2605e219f311920bf3fe2b031

data/context/best-practices.md

@@ -7,7 +7,7 @@ This guide gives an overview of best practices for using Async.
 `Async{}` has two uses: it creates an event loop if one doesn't exist, and it creates a task which runs asynchronously with respect to the parent scope. However, the top level `Async{}` block will be synchronous because it creates the event loop. In some programs, you do not care about executing asynchronously, but you still want your code to run in an event loop. `Sync{}` exists to do this efficiently.
 
 ```ruby
-require 'async'
+require "async"
 
 class Packages
 	def initialize(urls)
@@ -55,12 +55,24 @@ This expresses the intent to the caller that this method should only be invoked
 
 ## Use barriers to manage unbounded concurrency
 
-Barriers provide a way to manage an unbounded number of tasks.
+Barriers provide a way to manage an unbounded number of tasks. The top-level `Barrier` method creates a barrier with built-in load management using an `Async::Idler`.
 
 ```ruby
-Async do
-	barrier = Async::Barrier.new
-	
+Barrier do |barrier|
+	items.each do |item|
+		barrier.async do
+			process(item)
+		end
+	end
+end
+```
+
+The barrier will automatically wait for all tasks to complete and stop any outstanding tasks when the block exits. By default, it uses an `Async::Idler` to prevent system overload by scheduling tasks when the system load is below 80%.
+
+If you want to process tasks in order of completion, you can explicitly call `wait` with a block:
+
+```ruby
+Barrier do |barrier|
 	items.each do |item|
 		barrier.async do
 			process(item)
@@ -71,50 +83,61 @@ Async do
 	barrier.wait do |task|
 		result = task.wait
 		# Do something with result.
-
+		
 		# If you don't want to wait for any more tasks you can break:
 		break
 	end
-	
-	# Or just wait for all tasks to finish:
-	barrier.wait # May raise an exception if a task failed.
-ensure
-	# Stop all outstanding tasks in the barrier:
-	barrier&.stop
+end
+```
+
+To disable load management (not recommended for unbounded concurrency), you can pass `parent: nil`:
+
+```ruby
+Barrier(parent: nil) do |barrier|
+	# No load management - creates tasks as fast as possible
+	items.each do |item|
+		barrier.async do
+			process(item)
+		end
+	end
 end
 ```
 
 ## Use a semaphore to limit the number of concurrent tasks
 
-Semaphores allow you to limit the level of concurrency to a fixed number of tasks:
+Semaphores allow you to limit the level of concurrency to a fixed number of tasks. When using semaphores with barriers, the barrier should be the root of your task hierarchy, and the semaphore should be a child of the barrier:
 
 ```ruby
-Async do |task|
-	barrier = Async::Barrier.new
+Barrier(parent: nil) do |barrier|
 	semaphore = Async::Semaphore.new(4, parent: barrier)
 	
-	# Since the semaphore.async may block, we need to run the work scheduling in a child task:
-	task.async do
-		items.each do |item|
-			semaphore.async do
-				process(item)
-			end
+	items.each do |item|
+		semaphore.async do
+			process(item)
 		end
 	end
-	
-	# Wait for all the work to complete:
-	barrier.wait
-ensure
-	# Stop all outstanding tasks in the barrier:
-	barrier&.stop
 end
 ```
 
-In general, the barrier should be the root of your task hierarchy, and the semaphore should be a child of the barrier. This allows you to manage the lifetime of all tasks created by the semaphore, and ensures that all tasks are stopped when the barrier is stopped.
+In this example, we use `parent: nil` for the barrier to disable load management, since the semaphore already provides concurrency control. The semaphore limits execution to 4 concurrent tasks, and the barrier ensures all tasks are stopped when the block exits.
 
 ### Idler
 
-Idlers are like semaphores but with a limit defined by current processor utilization. In other words, an idler will do work up to a specific ratio of idle/busy time in the scheduler, and try to maintain that.
+Idlers are like semaphores but with a limit defined by current processor utilization. In other words, an idler will schedule work up to a specific ratio of idle/busy time in the scheduler.
+
+The top-level `Barrier` method uses an idler by default, making it safe for unbounded concurrency:
+
+```ruby
+Barrier do |barrier|  # Uses Async::Idler.new(0.8) by default
+	work.each do |work|
+		barrier.async do
+			work.call
+		end
+	end
+end
+```
+
+You can also use an idler directly without a barrier:
 
 ```ruby
 Async do
@@ -145,7 +168,7 @@ Async do |task|
 		while chunk = socket.gets
 			queue.push(chunk)
 		end
-	end
+		
 		# After this point, we won't be able to add items to the queue, and popping items will eventually result in nil once all items are dequeued:
 		queue.close
 	end
@@ -184,5 +207,3 @@ end
 ```
 
 It can be especially important to impose timeouts when processing user-provided data.
-
-## 

data/context/debugging.md

@@ -9,7 +9,7 @@ This guide explains how to debug issues with programs that use Async.
 The simplest way to debug an Async program is to use `puts` to print messages to the console. This is useful for understanding the flow of your program and the values of variables. However, it can be difficult to use `puts` to debug programs that use asynchronous code, as the output may be interleaved. To prevent this, wrap it in `Fiber.blocking{}`:
 
 ```ruby
-require 'async'
+require "async"
 
 Async do
 	3.times do |i|
@@ -42,8 +42,8 @@ If you don't use `Fiber.blocking{}`, the event loop will continue to run and you
 The `async-debug` gem provides a visual debugger for Async programs. It is a powerful tool that allows you to inspect the state of your program and see the hierarchy of your program:
 
 ```ruby
-require 'async'
-require 'async/debug'
+require "async"
+require "async/debug"
 
 Sync do
 	debugger = Async::Debug.serve

data/context/getting-started.md

@@ -65,7 +65,7 @@ You should consider the boundary around your program and the request handling. F
 Similar to a promise, {ruby Async::Task} produces results. In order to wait for these results, you must invoke {ruby Async::Task#wait}:
 
 ``` ruby
-require 'async'
+require "async"
 
 task = Async do
 	rand
@@ -99,7 +99,7 @@ end
 Unless you need fan-out, map-reduce style concurrency, you can actually use a slightly more efficient {ruby Kernel::Sync} execution model. This method will run your block in the current event loop if one exists, or create an event loop if not. You can use it for code which uses asynchronous primitives, but itself does not need to be asynchronous with respect to other tasks.
 
 ```ruby
-require 'async/http/internet'
+require "async/http/internet"
 
 def fetch(url)
 	Sync do
@@ -109,11 +109,11 @@ def fetch(url)
 end
 
 # At the level of your program, this method will create an event loop:
-fetch(...)
+fetch("https://example.com")
 
 Sync do
 	# The event loop already exists, and will be reused:
-	fetch(...)
+	fetch("https://example.com")
 end
 ```
 
@@ -154,13 +154,13 @@ The former allows you to inject the parent, which could be a barrier or semaphor
 The Fiber Scheduler interface is compatible with most pure Ruby code and well-behaved C code. For example, you can use {ruby Net::HTTP} for performing concurrent HTTP requests:
 
 ```ruby
-urls = [...]
+urls = ["http://example.com", "http://example.org", "http://example.net"]
 
 Async do
 	# Perform several concurrent requests:
 	responses = urls.map do |url|
 		Async do
-			Net::HTTP.get(url)
+			Net::HTTP.get(URI(url))
 		end
 	end.map(&:wait)
 end

data/context/scheduler.md

@@ -38,7 +38,7 @@ Async do |task|
 	task.print_hierarchy($stderr)
 	
 	# Kill the subtask
-	subtask.stop
+	subtask.cancel
 end
 ~~~
 
@@ -69,16 +69,16 @@ end
 
 You can use this approach to embed the reactor in another event loop. For some integrations, you may want to specify the maximum time to wait to {ruby Async::Scheduler#run_once}.
 
-### Stopping a Scheduler
+### Cancelling a Scheduler
 
-{ruby Async::Scheduler#stop} will stop the current scheduler and all children tasks.
+{ruby Async::Scheduler#cancel} will cancel the current scheduler and all children tasks.
 
 ### Fiber Scheduler Integration
 
 In order to integrate with native Ruby blocking operations, the {ruby Async::Scheduler} uses a {ruby Fiber::Scheduler} interface.
 
 ```ruby
-require 'async'
+require "async"
 
 scheduler = Async::Scheduler.new
 Fiber.set_scheduler(scheduler)

data/context/tasks.md

@@ -4,7 +4,7 @@ This guide explains how asynchronous tasks work and how to use them.
 
 ## Overview
 
-Tasks are the smallest unit of sequential code execution in {ruby Async}. Tasks can create other tasks, and Async tracks the parent-child relationship between tasks. When a parent task is stopped, it will also stop all its children tasks. The reactor always starts with one root task.
+Tasks are the smallest unit of sequential code execution in {ruby Async}. Tasks can create other tasks, and Async tracks the parent-child relationship between tasks. When a parent task is cancelled, it will also cancel all its children tasks. The reactor always starts with one root task.
 
 ```mermaid
 graph LR
@@ -23,11 +23,11 @@ graph LR
 
 A fiber is a lightweight unit of execution that can be suspended and resumed at specific points. After a fiber is suspended, it can be resumed later at the same point with the same execution state. Because only one fiber can execute at a time, they are often referred to as a mechanism for cooperative concurrency.
 
-A task provides extra functionality on top of fibers. A task behaves like a promise: it either succeeds with a value or fails with an exception. Tasks keep track of their parent-child relationships, and when a parent task is stopped, it will also stop all its children tasks. This makes it easier to create complex programs with many concurrent tasks.
+A task provides extra functionality on top of fibers. A task behaves like a promise: it either succeeds with a value or fails with an exception. Tasks keep track of their parent-child relationships, and when a parent task is cancelled, it will also cancel all its children tasks. This makes it easier to create complex programs with many concurrent tasks.
 
 ### Why does Async manipulate tasks and not fibers?
 
-The {ruby Async::Scheduler} actually works directly with fibers for most operations and isn't aware of tasks. However, the reactor does maintain a tree of tasks for the purpose of managing task and reactor life-cycle. For example, stopping a parent task will stop all its children tasks, and the reactor will exit when all tasks are finished.
+The {ruby Async::Scheduler} actually works directly with fibers for most operations and isn't aware of tasks. However, the reactor does maintain a tree of tasks for the purpose of managing task and reactor life-cycle. For example, cancelling a parent task will cancel all its children tasks, and the reactor will exit when all tasks are finished.
 
 ## Task Lifecycle
 
@@ -40,20 +40,20 @@ stateDiagram-v2
     
     running --> failed : unhandled StandardError-derived exception
     running --> complete : user code finished
-    running --> stopped : stop
+    running --> cancelled : cancel
 
-    initialized --> stopped : stop
+    initialized --> cancelled : cancel
     
     failed --> [*]
     complete --> [*]
-    stopped --> [*]
+    cancelled --> [*]
 ```
 
-Tasks are created in the `initialized` state, and are run by the reactor. During the execution, a task can either `complete` successfully, become `failed` with an unhandled `StandardError`-derived exception, or be explicitly `stopped`. In all of these cases, you can wait for a task to complete by using {ruby Async::Task#wait}.
+Tasks are created in the `initialized` state, and are run by the reactor. During the execution, a task can either `complete` successfully, become `failed` with an unhandled `StandardError`-derived exception, or be explicitly `cancelled`. In all of these cases, you can wait for a task to complete by using {ruby Async::Task#wait}.
 
 1. In the case the task successfully completed, the result will be whatever value was generated by the last expression in the task.
 2. In the case the task failed with an unhandled `StandardError`-derived exception, waiting on the task will re-raise the exception.
-3. In the case the task was stopped, the result will be `nil`.
+3. In the case the task was cancelled, the result will be `nil`.
 
 ## Starting A Task
 
@@ -175,8 +175,8 @@ Async do
 			break if done.size >= 2
 		end
 	ensure
-		# The remainder of the tasks will be stopped:
-		barrier.stop
+		# The remainder of the tasks will be cancelled:
+		barrier.cancel
 	end
 end
 ```
@@ -189,23 +189,28 @@ end
 barrier = Async::Barrier.new
 semaphore = Async::Semaphore.new(2, parent: barrier)
 
-jobs.each do |job|
-	semaphore.async(parent: barrier) do
-		# ... process job ...
+begin
+	jobs.each do |job|
+		semaphore.async do
+			# ... process job ...
+		end
 	end
-end
 
-# Wait until all jobs are done:
-barrier.wait
+	# Wait until all jobs are done:
+	barrier.wait
+ensure
+	# Cancel any remaining jobs:
+	barrier.cancel
+end
 ~~~
 
-## Stopping a Task
+## Cancelling a Task
 
 When a task completes execution, it will enter the `complete` state (or the `failed` state if it raises an unhandled exception).
 
-There are various situations where you may want to stop a task ({ruby Async::Task#stop}) before it completes. The most common case is shutting down a server. A more complex example is this: you may fan out multiple (10s, 100s) of requests, wait for a subset to complete (e.g. the first 5 or all those that complete within a given deadline), and then stop (terminate/cancel) the remaining operations.
+There are various situations where you may want to cancel a task ({ruby Async::Task#cancel}) before it completes. The most common case is shutting down a server. A more complex example is this: you may fan out multiple (10s, 100s) of requests, wait for a subset to complete (e.g. the first 5 or all those that complete within a given deadline), and then cancel the remaining operations.
 
-Using the above program as an example, let's stop all the tasks just after the first one completes.
+Using the above program as an example, let's cancel all the tasks just after the first one completes.
 
 ```ruby
 Async do
@@ -215,15 +220,15 @@ Async do
 			puts "Hello World #{i}"
 		end
 	end
-
-	# Stop all the above tasks:
-	tasks.each(&:stop)
+	
+	# Cancel all the above tasks:
+	tasks.each(&:cancel)
 end
 ```
 
-### Stopping all Tasks held in a Barrier
+### Cancelling all Tasks held in a Barrier
 
-To stop (terminate/cancel) all the tasks held in a barrier:
+To cancel all the tasks held in a barrier:
 
 ```ruby
 barrier = Async::Barrier.new
@@ -236,11 +241,11 @@ Async do
 		end
 	end
 	
-	barrier.stop
+	barrier.cancel
 end
 ```
 
-Unless your tasks all rescue and suppresses `StandardError`-derived exceptions, be sure to call ({ruby Async::Barrier#stop}) to stop the remaining tasks:
+Unless your tasks all rescue and suppresses `StandardError`-derived exceptions, be sure to call ({ruby Async::Barrier#cancel}) to cancel the remaining tasks:
 
 ```ruby
 barrier = Async::Barrier.new
@@ -256,7 +261,7 @@ Async do
 	begin
 		barrier.wait
 	ensure
-		barrier.stop
+		barrier.cancel
 	end
 end
 ```
@@ -268,10 +273,10 @@ In order to ensure your resources are cleaned up correctly, make sure you wrap r
 ~~~ ruby
 Async do
 	begin
-		socket = connect(remote_address) # May raise Async::Stop
+		socket = connect(remote_address) # May raise Async::Cancel
 
-		socket.write(...) # May raise Async::Stop
-		socket.read(...) # May raise Async::Stop
+		socket.write(...) # May raise Async::Cancel
+		socket.read(...) # May raise Async::Cancel
 	ensure
 		socket.close if socket
 	end
@@ -393,9 +398,9 @@ end
 
 Transient tasks are similar to normal tasks, except for the following differences:
 
-1. They are not considered by {ruby Async::Task#finished?}, so they will not keep the reactor alive. Instead, they are stopped (with a {ruby Async::Stop} exception) when all other (non-transient) tasks are finished.
+1. They are not considered by {ruby Async::Task#finished?}, so they will not keep the reactor alive. Instead, they are cancelled (with a {ruby Async::Cancel} exception) when all other (non-transient) tasks are finished.
 2. As soon as a parent task is finished, any transient child tasks will be moved up to be children of the parent's parent. This ensures that they never keep a sub-tree alive.
-3. Similarly, if you `stop` a task, any transient child tasks will be moved up the tree as above rather than being stopped.
+3. Similarly, if you `cancel` a task, any transient child tasks will be moved up the tree as above rather than being cancelled.
 
 The purpose of transient tasks is when a task is an implementation detail of an object or instance, rather than a concurrency process. Some examples of transient tasks:
 
@@ -409,8 +414,8 @@ and you could be handling 1000s of requests per second.
 The task doing the updating in the background is an implementation detail, so it is marked as `transient`.
 
 ```ruby
-require 'async'
-require 'thread/local' # thread-local gem.
+require "async"
+require "thread/local" # thread-local gem.
 
 class TimeStringCache
 	extend Thread::Local # defines `instance` class method that lazy-creates a separate instance per thread
@@ -434,7 +439,7 @@ class TimeStringCache
 				sleep(1)
 			end
 		ensure
-			# When the reactor terminates all tasks, `Async::Stop` will be raised from `sleep` and  this code will be invoked. By clearing `@refresh`, we ensure that the task will be recreated if needed again:
+			# When the reactor terminates all tasks, `Async::Cancel` will be raised from `sleep` and  this code will be invoked. By clearing `@refresh`, we ensure that the task will be recreated if needed again:
 			@refresh = nil
 		end
 	end
@@ -445,4 +450,4 @@ Async do
 end
 ```
 
-Upon existing the top level async block, the {ruby @refresh} task will be set to `nil`. Bear in mind, you should not share these resources across threads; doing so would need some form of mutual exclusion.
+Upon exiting the top level async block, the {ruby @refresh} task will be set to `nil`. Bear in mind, you should not share these resources across threads; doing so would need some form of mutual exclusion.