llm.rb 16.0.0.pre.8 → 16.0.0.pre.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d91d2b022e12ba720029c2083969db3438a9c449e733caa6a992a63baacde3b8
-  data.tar.gz: 498c7b90c52d7d642bfec0baa89ea27b86a5d9e6bf3e5eefb0abfebf8dc03d72
+  metadata.gz: 05acffd1b6e5349176d0f54466c43f26d7918aa7132de40f99b57ccce25c1bf4
+  data.tar.gz: 0cd1c8e985a857b207e8024d7fa65c84bd5ae3de742f11bf761683542d8eb637
 SHA512:
-  metadata.gz: 9a5d53788c58de77e215369a3a6fd77485e8fd8fccc1107218fac2d5f6bbc417dd44ccba9e37344ffa9084ded1949c33c516f708f6f6d50aaf6a5baacf8fec2b
-  data.tar.gz: 3d29709c6a8fd38b3631464adb21c22a7bfa491f5a7e365f38b52a2f2a6c07da3f63443e27530de19c2ce5446ce35a778b1f200cfcef4b554e65ef8bf1ee35f7
+  metadata.gz: db1d654420ee79781bcb296b0b975005b38199d7c0fabd0d83dbbe6392648bd6c3c6f41b701132ddbb4ae1b87289e9edce12b4c588b1c79736275944b4c4d279
+  data.tar.gz: b71bd520cdf43918b5223dd6a78985f174cbb07dd3f8b03b1d62ee4f4b2a178e39b0af4b4157d4db6d9f038b8418b330363b8c81b226125d30993ca9d090bf3b

data/CHANGELOG.md

@@ -27,7 +27,7 @@ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
 | Ruby 3.3 or later | Ruby 3.4 or later |
 | `record.messages` returns the messages the runtime holds | a `:jsonb` record returns a relation of `LLM::ActiveRecord::Message` rows |
 | a `set_tracer` method on an ORM model | `set tracer:` in the block, or `tracer:` on the wrapper |
-| a `:thread` or `:fiber` cancel that arrives before the tool starts is dropped | the cancel is held and raised once the tool starts |
+| a pre-start cancel is dropped on `:thread`, `:fiber`, and `:async`, a `NoMethodError` on `:fork`, and says nothing on `:ractor` | the cancel is held, and delivered inside the tool's call |
 | a cancel that arrives between two requests does nothing | the turn ends there, and the caller sees `LLM::Interrupt` |
 | a bare `rescue`, or `rescue => ex`, catches a cancel | `LLM::Interrupt` is outside `StandardError`, so both forms pass it through |
 
@@ -43,12 +43,16 @@ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
   that again with `#messages!`, and order the relation by `position`
   because it is unordered.
 
-* **function: hold a cancel on `:thread` and `:fiber` instead of dropping it** <br>
-  A cancel that arrived before the tool started used to be dropped, and the call
-  ran to completion. It is now held and raised once the tool starts, so a caller
-  that cancelled a task it had not yet spawned sees
-  [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html) where
-  it used to see a return.
+* **function: hold a cancel that arrives before the tool starts** <br>
+  A cancel that arrived before the tool started used to be lost: dropped on
+  `:thread`, `:fiber`, and `:async`, a `NoMethodError` on a `:fork` task that
+  had not been spawned, and nothing at all on `:ractor`. It is now held until
+  the call opens and delivered inside it, so the tool is entered, its own
+  `rescue` sees the interrupt, and a caller that cancelled a task it had not
+  yet spawned is answered with the cancel rather than a return. On `:async`
+  and `:fiber` the raise is asked of the scheduler rather than issued, so a
+  tool that never yields still answers, and that caller sees a return with the
+  tool told it was asked about.
 
 * **interrupt: reach a turn that is between its requests** <br>
   A cancel that arrived between two requests used to do nothing, because
@@ -110,42 +114,50 @@ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
 ### Function
 
 * **function: deliver an interrupt to the tool, not to whatever is running** <br>
-  A cancel now raises
-  [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html) on the
-  thread running the tool, and only while it runs: arriving early, it is held
-  until the tool starts, so the tool's own `rescue` sees it; arriving late, it
-  is a no-op instead of throwing the result away. Applies to `:fork` and
-  `:ractor`.
-
-* **function: answer a ractor-backed task once its ractor has gone** <br>
-  A `:ractor` tool's result is now held by the task instead of asked of the
-  ractor that ran it, so a late wait is answered, a second wait comes from
-  memory, and a cancel returns `nil` instead of raising. A group's cancel now
-  reaches every task, not just the ones still running.
-
-* **function: tell an `:async` tool it was cancelled** <br>
-  A cancel now raises `LLM::Interrupt` on the fiber the tool runs in, so the
-  tool is told and can clean up. Before, the cancel was queued and the tool
-  kept running, its result written to a queue nobody read.
-
-* **function: run the interrupt hook on every strategy** <br>
-  A tool that implements
+  A cancel is now aimed at the tool on every concurrency strategy. A running
+  tool is entered and
+  [`LLM::Interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Interrupt.html) raised
+  inside it, so its own `rescue` and its
   [`LLM::Tool#on_interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#on_interrupt-instance_method)
-  is now told on every concurrency strategy. Before, `:async`, `:fork`, and
-  `:ractor` never ran it, and a class-backed tool ran it on no strategy at all.
-
-* **function: hold a cancel that arrives before a thread or fiber starts** <br>
-  Fix a bug where a cancel that arrived before a `:thread` or `:fiber` tool
-  started was dropped, and the call ran to completion as if nobody had asked it
-  to stop. The cancel is now held on the task and raised once the body is live,
-  so the tool's `on_interrupt` hook runs and the caller sees `LLM::Interrupt`.
+  hook see it, whether the tool was given as an instance or a class. A cancel
+  that arrives before the tool starts is held and delivered inside the call, and
+  one that arrives after the tool returns is a no-op that leaves the result
+  alone. Before, the raise landed wherever the target thread or fiber had got
+  to, and a tool's hook ran on some strategies but not others. A pre-start
+  cancel was dropped on `:thread`, `:fiber`, and `:async`, was a `NoMethodError`
+  on `:fork`, and said nothing on `:ractor`. Three strategies keep a shape of
+  their own: `:fiber` and `:async` ask the fiber scheduler for the raise, so a
+  tool that never yields is one it cannot reach, and `:sequential` runs the tool
+  in the caller's own thread, so it tells the tool through its hook alone.
 
 * **function: answer a task that has already answered** <br>
-  A second `#wait` on an `:async` or `:fork` task is answered from what the
-  first one took, which is what `Thread#value` does, so a task that has answered
-  answers again. Before, a second wait blocked on a queue nothing would fill, or
-  read a channel the task had already closed. An interrupt re-raises the same
-  exception rather than blocking.
+  A second `#wait` is answered from what the first one took, which is what
+  `Thread#value` does, so a task that has answered answers again. That covers an
+  `:async` queue the first wait drained, a `:fork` channel the task closed, and
+  a `:ractor` whose result is held by a ractor of the task's own rather than
+  asked of the ractor that ran the tool and has gone. A group's cancel reaches
+  every task it holds, and an interrupt re-raises the same exception rather than
+  blocking. Before, a second wait blocked on a queue nothing would fill, or read
+  a channel or ractor that had already closed.
+
+* **function: answer a forked call that ended without a result** <br>
+  A `:fork` tool that died before it wrote anything used to leave
+  [`LLM::Function::Fork::Task#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Fork/Task.html#wait-instance_method)
+  blocked on a read that never ended, because this side still held the write
+  end of the same channel and so never saw end of file. Each side now closes
+  the end it does not use, which makes that ending an `EOFError`, and `#wait`
+  answers it in band rather than raising into the turn, the way the runtime
+  answers a tool that raised: `{error: true, type: "EOFError", message: "the
+  tool exited unexpectedly"}`. A second wait is given the same return.
+
+* **function: refuse a `:fiber` scheduler that cannot hold a cancel** <br>
+  The `:fiber` strategy now raises
+  [`LLM::FiberError`](https://r.uby.dev/api-docs/llm.rb/LLM/FiberError.html) when
+  `Fiber.scheduler` does not implement `fiber_interrupt`, because a cancel that
+  arrives before the call is held by asking the scheduler for a raise at the
+  call's first instruction. A scheduler that cannot be asked would be sent the
+  cancel before the call instead, where the tool never runs, so the strategy
+  refuses in the caller's hands, before a fiber is scheduled at all.
 
 ### ORM
 
@@ -181,12 +193,26 @@ Releases before v15 are kept in [changelog/old.md](changelog/old.md).
   because DeepSeek has no Files API, and a non-image local file raises because
   its models read images only.
 
+* **provider: keep a scoped header alive through garbage collection** <br>
+  Fix a bug where a header set by
+  [`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method)
+  could be collected before its block returned, so a garbage collection inside
+  the block was enough to make a request later in the same scope go out without
+  it. The store held the header hash weakly. It now holds it for as long as the
+  provider is alive, so a scoped header lasts as long as the scope does.
+
 ### Registry
 
 * **refresh model metadata** <br>
-  Update `data/` with current listings, limits, and pricing. Alibaba, OpenAI,
-  Bedrock, DeepInfra, Google, and OpenRouter gain models and correct limits or
-  prices, and OpenRouter drops six entries.
+  Update `data/` with current listings, limits, and pricing. Anthropic adds
+  Claude Sonnet 5.5, OpenAI adds GPT-6.1 Sol and the Daybreak Blue and Daybreak
+  Red models, Bedrock adds Claude Sonnet 5.5, GPT-6 Sol, GPT-6 Luna, and GPT-6.1
+  Sol across regions plus Grok 4.7 in the global and US regions, Alibaba adds
+  the Qwen3.5, Qwen3.7, and Qwen3.8 Flash models, DeepInfra adds the Xiaomi
+  MiMo V2.6 models and `tencent/Hy4-preview`, xAI adds Grok Imagine Video 1.5
+  Lite, and OpenRouter adds ten models. OpenRouter drops six entries, Mistral
+  drops `magistral-small`, and Google, DeepSeek, Moonshot, and Z.ai correct
+  limits and prices.
 
 ### Schema
 

data/data/bedrock.json

@@ -371,6 +371,23 @@
         "input": 922000,
         "output": 128000
       },
+      "experimental": {
+        "modes": {
+          "ultrafast": {
+            "cost": {
+              "input": 60,
+              "output": 300,
+              "cache_read": 6,
+              "cache_write": 75
+            },
+            "provider": {
+              "body": {
+                "service_tier": "ultrafast"
+              }
+            }
+          }
+        }
+      },
       "cost": {
         "input": 10,
         "output": 50,
@@ -397,6 +414,72 @@
       },
       "canonical_model_id": "openai/gpt-6-astra"
     },
+    "us.openai.gpt-6.1-sol": {
+      "id": "us.openai.gpt-6.1-sol",
+      "name": "GPT-6.1 Sol (US)",
+      "description": "Near-Astra performance for complex coding, computer use, and professional work at a lower cost.",
+      "family": "gpt-sol",
+      "attachment": true,
+      "reasoning": true,
+      "reasoning_options": [
+        {
+          "type": "effort",
+          "values": [
+            "low",
+            "medium",
+            "high",
+            "xhigh",
+            "max"
+          ]
+        }
+      ],
+      "tool_call": true,
+      "structured_output": true,
+      "temperature": false,
+      "knowledge": "2026-04-30",
+      "release_date": "2026-09-29",
+      "last_updated": "2026-09-29",
+      "modalities": {
+        "input": [
+          "text",
+          "image"
+        ],
+        "output": [
+          "text"
+        ]
+      },
+      "open_weights": false,
+      "limit": {
+        "context": 1050000,
+        "input": 922000,
+        "output": 128000
+      },
+      "cost": {
+        "input": 2.2,
+        "output": 11,
+        "cache_read": 0.11,
+        "cache_write": 2.75,
+        "tiers": [
+          {
+            "input": 4.4,
+            "output": 16.5,
+            "cache_read": 0.22,
+            "cache_write": 5.5,
+            "tier": {
+              "type": "context",
+              "size": 272000
+            }
+          }
+        ],
+        "context_over_200k": {
+          "input": 4.4,
+          "output": 16.5,
+          "cache_read": 0.22,
+          "cache_write": 5.5
+        }
+      },
+      "canonical_model_id": "openai/gpt-6.1-sol"
+    },
     "eu.amazon.nova-lite-v1:0": {
       "id": "eu.amazon.nova-lite-v1:0",
       "name": "Nova Lite (EU)",
@@ -2379,6 +2462,78 @@
       },
       "canonical_model_id": "nvidia/nemotron-nano-12b-v2-vl"
     },
+    "openai.gpt-6.1-sol": {
+      "id": "openai.gpt-6.1-sol",
+      "name": "GPT-6.1 Sol",
+      "description": "Near-Astra performance for complex coding, computer use, and professional work at a lower cost.",
+      "family": "gpt-sol",
+      "attachment": true,
+      "reasoning": true,
+      "reasoning_options": [
+        {
+          "type": "effort",
+          "values": [
+            "low",
+            "medium",
+            "high",
+            "xhigh",
+            "max"
+          ]
+        }
+      ],
+      "tool_call": true,
+      "structured_output": true,
+      "temperature": false,
+      "knowledge": "2026-04-30",
+      "release_date": "2026-09-29",
+      "last_updated": "2026-09-29",
+      "modalities": {
+        "input": [
+          "text",
+          "image",
+          "pdf"
+        ],
+        "output": [
+          "text"
+        ]
+      },
+      "open_weights": false,
+      "limit": {
+        "context": 1050000,
+        "input": 922000,
+        "output": 128000
+      },
+      "provider": {
+        "npm": "@ai-sdk/amazon-bedrock/mantle",
+        "api": "https://bedrock-mantle.${AWS_REGION}.api.aws/openai/v1",
+        "shape": "responses"
+      },
+      "cost": {
+        "input": 2.2,
+        "output": 11,
+        "cache_read": 0.11,
+        "cache_write": 2.75,
+        "tiers": [
+          {
+            "input": 4.4,
+            "output": 16.5,
+            "cache_read": 0.22,
+            "cache_write": 5.5,
+            "tier": {
+              "type": "context",
+              "size": 272000
+            }
+          }
+        ],
+        "context_over_200k": {
+          "input": 4.4,
+          "output": 16.5,
+          "cache_read": 0.22,
+          "cache_write": 5.5
+        }
+      },
+      "canonical_model_id": "openai/gpt-6.1-sol"
+    },
     "us.amazon.nova-2-lite-v1:0": {
       "id": "us.amazon.nova-2-lite-v1:0",
       "name": "Nova 2 Lite (US)",
@@ -2468,6 +2623,23 @@
         "input": 922000,
         "output": 128000
       },
+      "experimental": {
+        "modes": {
+          "ultrafast": {
+            "cost": {
+              "input": 66,
+              "output": 330,
+              "cache_read": 6.6,
+              "cache_write": 82.5
+            },
+            "provider": {
+              "body": {
+                "service_tier": "ultrafast"
+              }
+            }
+          }
+        }
+      },
       "cost": {
         "input": 11,
         "output": 55,
@@ -2567,6 +2739,72 @@
       },
       "canonical_model_id": "openai/gpt-5.6-terra"
     },
+    "global.openai.gpt-6.1-sol": {
+      "id": "global.openai.gpt-6.1-sol",
+      "name": "GPT-6.1 Sol (Global)",
+      "description": "Near-Astra performance for complex coding, computer use, and professional work at a lower cost.",
+      "family": "gpt-sol",
+      "attachment": true,
+      "reasoning": true,
+      "reasoning_options": [
+        {
+          "type": "effort",
+          "values": [
+            "low",
+            "medium",
+            "high",
+            "xhigh",
+            "max"
+          ]
+        }
+      ],
+      "tool_call": true,
+      "structured_output": true,
+      "temperature": false,
+      "knowledge": "2026-04-30",
+      "release_date": "2026-09-29",
+      "last_updated": "2026-09-29",
+      "modalities": {
+        "input": [
+          "text",
+          "image"
+        ],
+        "output": [
+          "text"
+        ]
+      },
+      "open_weights": false,
+      "limit": {
+        "context": 1050000,
+        "input": 922000,
+        "output": 128000
+      },
+      "cost": {
+        "input": 2,
+        "output": 10,
+        "cache_read": 0.1,
+        "cache_write": 2.5,
+        "tiers": [
+          {
+            "input": 4,
+            "output": 15,
+            "cache_read": 0.2,
+            "cache_write": 5,
+            "tier": {
+              "type": "context",
+              "size": 272000
+            }
+          }
+        ],
+        "context_over_200k": {
+          "input": 4,
+          "output": 15,
+          "cache_read": 0.2,
+          "cache_write": 5
+        }
+      },
+      "canonical_model_id": "openai/gpt-6.1-sol"
+    },
     "anthropic.claude-opus-4-6-v1": {
       "id": "anthropic.claude-opus-4-6-v1",
       "name": "Claude Opus 4.6",
@@ -7092,6 +7330,23 @@
         "input": 922000,
         "output": 128000
       },
+      "experimental": {
+        "modes": {
+          "ultrafast": {
+            "cost": {
+              "input": 66,
+              "output": 330,
+              "cache_read": 6.6,
+              "cache_write": 82.5
+            },
+            "provider": {
+              "body": {
+                "service_tier": "ultrafast"
+              }
+            }
+          }
+        }
+      },
       "provider": {
         "npm": "@ai-sdk/amazon-bedrock/mantle",
         "api": "https://bedrock-mantle.${AWS_REGION}.api.aws/openai/v1",

data/data/deepinfra.json

@@ -166,6 +166,7 @@
         "context": 1048576,
         "output": 16384
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.2,
         "output": 0.8
@@ -541,8 +542,8 @@
         "output": 32768
       },
       "cost": {
-        "input": 0.13,
-        "output": 0.38
+        "input": 0.15,
+        "output": 0.4
       },
       "canonical_model_id": "google/gemma-4-31b-it"
     },
@@ -615,6 +616,7 @@
         "context": 131072,
         "output": 8192
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.02,
         "output": 0.1
@@ -886,7 +888,8 @@
       },
       "cost": {
         "input": 0.1,
-        "output": 0.95
+        "output": 0.95,
+        "cache_read": 0.1
       }
     },
     "Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo": {
@@ -1135,6 +1138,7 @@
         "context": 262144,
         "output": 65536
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.29,
         "output": 2.4
@@ -1793,6 +1797,7 @@
         "context": 262144,
         "output": 256000
       },
+      "status": "deprecated",
       "cost": {
         "input": 0.2,
         "output": 1.15,
@@ -2110,6 +2115,7 @@
         "context": 202752,
         "output": 16384
       },
+      "status": "deprecated",
       "cost": {
         "input": 1.05,
         "output": 3.5,

data/data/deepseek.json

@@ -153,10 +153,10 @@
         "output": 393216
       },
       "cost": {
-        "input": 0.435,
-        "output": 0.87,
-        "reasoning": 0.87,
-        "cache_read": 0.003625
+        "input": 0.66,
+        "output": 1.98,
+        "reasoning": 1.98,
+        "cache_read": 0.022
       },
       "canonical_model_id": "deepseek/deepseek-v4-pro-0813"
     },

data/data/moonshot.json

@@ -16,9 +16,6 @@
       "attachment": true,
       "reasoning": true,
       "reasoning_options": [
-        {
-          "type": "toggle"
-        },
         {
           "type": "effort",
           "values": [