llm.rb 16.0.0.pre.7 → 16.0.0.pre.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0a92555a0d4e73fd37604103300639f048607b332a106a042615ff76880b9665
-  data.tar.gz: de618250fdd9cdf37c928e2753e6be173201f482ce30ae97ce0e080bea3c8a24
+  metadata.gz: d91d2b022e12ba720029c2083969db3438a9c449e733caa6a992a63baacde3b8
+  data.tar.gz: 498c7b90c52d7d642bfec0baa89ea27b86a5d9e6bf3e5eefb0abfebf8dc03d72
 SHA512:
-  metadata.gz: 8739cb9db9168ea2a6daedb82a068b51085509ffc0447e0d4f7415fc7b3c02051d9a26b85f3a0a0b7cda7e7cb4ae47f50f5a7f181a56b1f97d516e4b34a99117
-  data.tar.gz: 6c615f5ac41d186a2215db2f3d64fe68896f960bab8154629ac3322666e350bbe29dce80622cac190f6e1b8b57827413545cc41d5098f7d88867f0a9469ced2a
+  metadata.gz: 9a5d53788c58de77e215369a3a6fd77485e8fd8fccc1107218fac2d5f6bbc417dd44ccba9e37344ffa9084ded1949c33c516f708f6f6d50aaf6a5baacf8fec2b
+  data.tar.gz: 3d29709c6a8fd38b3631464adb21c22a7bfa491f5a7e365f38b52a2f2a6c07da3f63443e27530de19c2ce5446ce35a778b1f200cfcef4b554e65ef8bf1ee35f7

data/CHANGELOG.md

@@ -29,6 +29,7 @@ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
 | a `set_tracer` method on an ORM model | `set tracer:` in the block, or `tracer:` on the wrapper |
 | a `:thread` or `:fiber` cancel that arrives before the tool starts is dropped | the cancel is held and raised once the tool starts |
 | a cancel that arrives between two requests does nothing | the turn ends there, and the caller sees `LLM::Interrupt` |
+| a bare `rescue`, or `rescue => ex`, catches a cancel | `LLM::Interrupt` is outside `StandardError`, so both forms pass it through |
 
 * **drop Ruby 3.3 support** <br>
   The gem now requires Ruby 3.4 or later. Ruby 3.3 cannot hold an
@@ -57,8 +58,19 @@ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
   first. `LLM::Agent#run_loop` records that caller for as long as the turn
   lasts, and it answers `interrupt!` through
   [`LLM::Agent::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent/Interrupt.html).
-  `LLM::Interrupt` is a `StandardError`, so a broad `rescue` catches it
-  wherever the turn had reached.
+  The raise lands in the thread or fiber the turn runs on, so it unwinds out
+  of the turn to whoever called it.
+
+* **interrupt: move `LLM::Interrupt` outside `StandardError`** <br>
+  `LLM::Interrupt` is now an `Exception` rather than an `LLM::Error`, so a
+  bare `rescue` and a `rescue => ex` pass a cancel through instead of
+  swallowing it. Before, a broad rescue in a tool, a tracer, or an
+  application could eat an interrupt, and the turn looked like one that had
+  ignored its cancel. It is not a `SignalException`, since an interrupt has
+  to be catchable and a signal is a framework's own condition. A caller that
+  handles a cancel names `LLM::Interrupt`; `#wait` and the tool's
+  `on_interrupt` hook are unchanged, and a task still carries the interrupt
+  back to the caller rather than raising it inside its reactor.
 
 ### ActiveRecord
 

data/docs/deepdive/advanced/cancellation.md

@@ -93,9 +93,12 @@ A cancel is delivered at the point the turn has reached, which can be
 any point of it: inside a stream callback, a compactor, a transformer,
 a guard, or the persistence write a step makes.
 [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html)
-is a `StandardError`, so a `rescue` that names no class catches it too.
-Code that has to see the end of a turn should re-raise what it does not
-recognize rather than absorb it.
+sits outside `StandardError`, so neither a bare `rescue` nor a
+`rescue => ex` catches it, and the language is what keeps a cancel from
+being eaten rather than a convention every rescue has to remember. Code
+that handles a cancel names `LLM::Interrupt`, the way the example above
+does; it is not an `LLM::Error` either, so a rescue of the runtime's
+error superclass does not catch it.
 
 The runtime's own path out of a turn does not swallow an interrupt:
 [`LLM::Context#try`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#try)

data/docs/deepdive/reference/tracer.md

@@ -157,13 +157,15 @@ the tool runs and returns the span that `on_tool_finish` and
 `on_tool_error` receive.
 
 An interrupt is neither a finish nor an error, and it is reported to a
-hook of its own. `on_interrupt` is called with a `scope:` of `:request`
-or `:tool` before the caller is given the exception, so a tracer that
-records what happened to a turn records it while the work is still in
-flight. The hook receives the span that `on_request_start` returned and
-its `request_id`; a tool pass carries neither, because a tool's span
-belongs to the call. A cancel reaches every tool that is running, so the
-tool phase is announced once rather than once per tool.
+hook of its own. `on_interrupt` is called with a `scope:` of `:request`,
+`:tool`, or `:agent` before the caller is given the exception, so a
+tracer that records what happened to a turn records it while the work is
+still in flight. The hook receives the span that `on_request_start`
+returned and its `request_id`; a tool pass carries neither, because a
+tool's span belongs to the call, and an `:agent` pass is one that landed
+between a turn's requests, where there was nothing more precise to
+interrupt. A cancel reaches every tool that is running, so the tool phase
+is announced once rather than once per tool.
 
 A tracer's own lifetime is bracketed as well. `on_exit` fires once,
 when the last scope that is open for that tracer ends. That scope can

data/lib/llm/error.rb

@@ -63,8 +63,19 @@ module LLM
   ContextWindowError = Class.new(InvalidRequestError)
 
   ##
-  # When a request is interrupted
-  Interrupt = Class.new(Error)
+  # When a request is interrupted.
+  #
+  # It sits outside `StandardError`, so that neither a bare `rescue` nor a
+  # `rescue => e` can swallow it by mistake: an interrupt is a request to
+  # stop, and a turn whose cancel was eaten looks like a turn that ignored
+  # one.
+  #
+  # It is not a `SignalException` either. A signal is a framework's own
+  # condition rather than a task's - RSpec re-raises one that escapes an
+  # example, and an async reactor ends its own thread rather than failing
+  # the task that raised it - and an interrupt has to be something a caller
+  # can catch.
+  Interrupt = Class.new(Exception)
 
   ##
   # When a concurrency strategy cannot execute a given tool

data/lib/llm/function/async/task.rb

@@ -49,13 +49,15 @@ module LLM::Function::Async
     # hook that runs after the queue was pushed is a hook the caller can
     # race past.
     #
-    # The rescue below is the other half of that promise, and it is written
-    # to catch everything rather than an interrupt. A hook that raises from
-    # the block's `ensure` unwinds past the push, so the queue would be left
-    # empty and `#wait` would wait on it forever - the outcome this comment
-    # already names. Whatever comes out of the block is pushed, and `#wait`
-    # hands anything that is an exception to the caller the way `Thread#value`
-    # and `Fiber#value` do for the other in-process strategies.
+    # `LLM::Interrupt` is a subclass of `Exception`, and one that is left to
+    # raise kills the reactor's thread and takes every other task on that
+    # reactor with it. So the task rescues it, stores it on its own queue,
+    # and exits silently - and the caller reads the queue and raises the
+    # interrupt on its own thread or fiber, which is where it was asked for.
+    #
+    # The rescue below names the interrupt rather than leaving it to a bare
+    # rescue, because a bare rescue - and `rescue => e` - reaches only
+    # `StandardError`.
     # @return [nil]
     def spawn
       return if @guarded
@@ -65,10 +67,22 @@ module LLM::Function::Async
         task = Async::Task.current
         @task = task
         @scheduler = Fiber.scheduler
-        raise LLM::Interrupt if @cancelled
+        ##
+        # A cancel that arrived before this ran. There is no tool to tell,
+        # so the caller is given it and the task ends - for the same reason
+        # the block below answers rather than raises.
+        if @cancelled
+          @queue << LLM::Interrupt.new
+          next
+        end
         task.defer_cancel do
           result = begin
             function.call
+          rescue LLM::Interrupt => ex
+            ##
+            # The interrupt is the block's result, and the queue carries it
+            # to the caller.
+            ex
           ensure
             ##
             # The hook runs on the reactor's thread rather than on the one
@@ -80,15 +94,15 @@ module LLM::Function::Async
           end
           @queue << result
         end
-      rescue => e
+      rescue LLM::Interrupt, StandardError => e
         ##
         # This runs after the `defer_cancel` block's `ensure`, so the tool is
-        # told first and the caller second. It answers an interrupt, a
-        # cancel that arrived before `spawn` - the block raises before it
-        # reaches a tool, and there is no result to push - and a hook whose
-        # own error unwound past the push.
+        # told first and the caller second. It answers a hook whose own
+        # error unwound past the push, and it is the backstop for an
+        # interrupt that arrived from somewhere other than the call above -
+        # which is why the interrupt is pushed and not raised on.
         @queue << e
-        raise
+        raise unless LLM::Interrupt === e
       end
       nil
     end

data/lib/llm/function/fiber/task.rb

@@ -72,9 +72,20 @@ module LLM::Function::Fiber
               raise LLM::Interrupt
             end
             @queue << function.call
-          rescue => ex
+          rescue LLM::Interrupt, StandardError => ex
+            ##
+            # The interrupt is stored on the queue rather than raised on.
+            # `LLM::Interrupt` is a subclass of `Exception`, and one that is
+            # left to raise kills the scheduler's thread and takes the tasks
+            # running on it with it - the caller reads the queue and raises
+            # the interrupt on its own thread or fiber, and this task exits
+            # silently.
+            #
+            # The rescue names the interrupt rather than leaving it to a bare
+            # rescue, because a bare rescue - and `rescue => ex` - reaches
+            # only `StandardError`.
             @queue << ex
-            raise
+            raise unless LLM::Interrupt === ex
           ensure
             ##
             # The hook runs on the fiber the call runs on, once the call has

data/lib/llm/tracer.rb

@@ -195,7 +195,7 @@ module LLM
     end
 
     ##
-    # Called when a request or a tool is interrupted.
+    # Called when a request, a tool, or a turn is interrupted.
     #
     # An interrupt is not a failure, so this is not {#on_request_error} or
     # {#on_tool_error} under another name: it is the third ending either
@@ -206,7 +206,9 @@ module LLM
     # A request is announced once, by the transport. A tool is announced
     # once for the phase rather than once per tool: a cancel reaches every
     # tool that is running and the caller hears one exception, so there is
-    # no one tool the announcement belongs to.
+    # no one tool the announcement belongs to. A turn is announced as
+    # `:agent` when the interrupt lands between two of its requests, which
+    # is the one point of a turn that has no span of its own.
     #
     # The default does nothing, which is the difference from the rest of the
     # lifecycle. Every other hook raises, because a tracer that means to draw
@@ -214,7 +216,8 @@ module LLM
     # whatever tracer happens to be bound, and a hook that raised here would
     # replace the interrupt every caller is written against.
     # @param [Symbol] scope
-    #  :request for a request, :tool for a tool
+    #  :request for a request, :tool for a tool, :agent for a turn that is
+    #  between its requests
     # @param [Object, nil] span
     #  The span that {#on_request_start} returned. A tool pass has none:
     #  a tool's span belongs to the call, and the announcement is not one

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "16.0.0.pre.7"
+  VERSION = "16.0.0.pre.8"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 16.0.0.pre.7
+  version: 16.0.0.pre.8
 platform: ruby
 authors:
 - Robert Gleeson