llm.rb 0.15.0 → 0.16.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 68bbdddd157d6df71729378a46b4759140d7fe13c44a1e14940a0a3a367d277c
-  data.tar.gz: f24aa6b6042b58857ca419c0039422f678e92355f5fdfd0064d2931108338866
+  metadata.gz: c46802e2152430f164990a91499be669e928201e7793b162f8f62152349078af
+  data.tar.gz: 58269b584d08e9d236a3d85ac5728f4e4453a845ace9ebbf8c1ac22890609a1f
 SHA512:
-  metadata.gz: e5043707425445ea5709f4eed33b3feab33a376a66569abe46b6ad93274ffd3044eef1099625ec75f24f5ee5fb5387493adc97332c7be1d6fa2e085d6e33281c
-  data.tar.gz: 6f60c60130904dd23c67d65d50a04614c32659d5fe7af5e37ec2f9d6f25ccda1712a22bf872a7ae4c4aae49c2755cb1adff593fe1435c4231f445c40761a6477
+  metadata.gz: 9003c9cac451081ce0589f54acb2d72813e53a8708426e320e30732aa5842d99187af14f1c54bab69cfcc3bcc401529375377547a090f530aa1da140fd0a4b3f
+  data.tar.gz: 691e4ebfd80fcfa7af22f1ca26393256a69ddf6957be8960590eebddc5e20a1a30db3ba8cfc1ae65dba10491df6f2844fd72d3ca2444420cb23db357a0225e25

data/README.md

@@ -1,7 +1,7 @@
 ## About
 
 llm.rb is a zero-dependency Ruby toolkit for Large Language Models that
-includes OpenAI, Gemini, Anthropic, xAI (grok), DeepSeek, Ollama, and
+includes OpenAI, Gemini, Anthropic, xAI (Grok), DeepSeek, Ollama, and
 LlamaCpp. The toolkit includes full support for chat, streaming, tool calling,
 audio, images, files, and structured outputs (JSON Schema).
 
@@ -9,9 +9,13 @@ audio, images, files, and structured outputs (JSON Schema).
 
 #### Demo
 
+This cool demo writes a new [llm-shell](https://github.com/llmrb/llm-shell#readme) command
+with the help of [llm.rb](https://github.com/llmrb/llm#readme). <br> Similar-ish to
+GitHub Copilot but for the terminal.
+
 <details>
-  <summary>Play</summary>
-  <img src="share/llm-shell/examples/demo.gif/">
+  <summary>Start demo</summary>
+  <img src="https://github.com/llmrb/llm/blob/main/share/llm-shell/examples/demo.gif?raw=true" alt="llm-shell demo" />
 </details>
 
 #### Guides
@@ -22,18 +26,40 @@ audio, images, files, and structured outputs (JSON Schema).
   a blog post that implements an age estimation tool
 * [How to edit an image with Gemini](https://0x1eef.github.io/posts/how-to-edit-images-with-gemini/) &ndash;
   a blog post that implements image editing with Gemini
+* [Fast sailing with persistent connections](https://0x1eef.github.io/posts/persistent-connections-with-llm.rb/) &ndash;
+  a blog post that optimizes performance with a thread-safe connection pool
 
 #### Ecosystem
 
 * [llm-shell](https://github.com/llmrb/llm-shell) &ndash; a developer-oriented console for Large Language Model communication
 * [llm-spell](https://github.com/llmrb/llm-spell) &ndash; a utility that can correct spelling mistakes with a Large Language Model
 
+#### Show code
+
+A simple chatbot that maintains a conversation and streams
+responses in real-time:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+bot = LLM::Bot.new(llm, stream: $stdout)
+loop do
+  print "> "
+  input = $stdin.gets&.chomp || break
+  bot.chat(input).flush
+  print "\n"
+end
+```
+
 ## Features
 
 #### General
 - ✅ A single unified interface for multiple providers
 - 📦 Zero dependencies outside Ruby's standard library
 - 🚀 Smart API design that minimizes the number of requests made
+- ♻️ Optional: per-provider, process-wide connection pool via net-http-persistent
 
 #### Chat, Agents
 - 🧠 Stateless and stateful chat via completions and responses API
@@ -110,13 +136,32 @@ llm = LLM.ollama(key: nil)
 llm = LLM.llamacpp(key: nil)
 ```
 
+#### Persistence
+
+The llm.rb library can maintain a process-wide connection pool
+for each provider that is instantiated. This feature can improve
+performance but it is optional, the implementation depends on
+[net-http-persistent](https://github.com/dbrain/net-http-persistent),
+and the gem should be installed separately:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm  = LLM.openai(key: ENV["KEY"], persistent: true)
+res1 = llm.responses.create "message 1"
+res2 = llm.responses.create "message 2", previous_response_id: res1.response_id
+res3 = llm.responses.create "message 3", previous_response_id: res2.response_id
+print res3.output_text, "\n"
+```
+
 ### Conversations
 
 #### Completions
 
 > This example uses the stateless chat completions API that all
 > providers support. A similar example for OpenAI's stateful
-> responses API is available in the [docs/](docs/OPENAI.md#responses)
+> responses API is available in the [docs/](https://0x1eef.github.io/x/llm.rb/file.OPENAI.html#responses)
 > directory.
 
 The following example creates an instance of
@@ -149,7 +194,8 @@ bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
 > There Is More Than One Way To Do It (TIMTOWTDI) when you are
 > using llm.rb &ndash; and this is especially true when it
 > comes to streaming. See the streaming documentation in
-> [docs/](docs/STREAMING.md#scopes) for more details.
+> [docs/](https://0x1eef.github.io/x/llm.rb/file.STREAMING.html#scopes)
+> for more details.
 
 The following example streams the messages in a conversation
 as they are generated in real-time. The `stream` option can
@@ -170,7 +216,7 @@ bot.chat(stream: $stdout) do |prompt|
   prompt.user ["Tell me about this URL", URI(url)]
   prompt.user ["Tell me about this PDF", File.open("handbook.pdf", "rb")]
   prompt.user "Are the URL and PDF similar to each other?"
-end.to_a
+end.flush
 ```
 
 ### Schema
@@ -263,6 +309,43 @@ bot.chat bot.functions.map(&:call) # report return value to the LLM
 # {stderr: "", stdout: "FreeBSD"}
 ```
 
+#### Provider
+
+The
+[LLM::Function](https://0x1eef.github.io/x/llm.rb/LLM/Function.html)
+class can define a local function that can be called by a provider on your behalf,
+and the
+[LLM::Tool](https://0x1eef.github.io/x/llm.rb/LLM/Tool.html)
+class represents a tool that is defined and implemented by a provider, and we can
+request that the provider call the tool on our behalf. That's the primary difference
+between a function implemented locally and a tool implemented by a provider. The
+available tools depend on the provider, and the following example uses the
+OpenAI provider to execute Python code on OpenAI's servers:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.responses.create "Run: 'print(\"hello world\")'", tools: [llm.tool(:code_interpreter)]
+print res.output_text, "\n"
+```
+
+#### Web Search
+
+A common tool among all providers is the ability to perform a web search, and
+the following example uses the OpenAI provider to search the web using the
+Web Search tool. This can also be done with the Anthropic and Gemini providers:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+res = llm.web_search(query: "summarize today's news")
+print res.output_text, "\n"
+```
+
 ### Files
 
 #### Create
@@ -504,6 +587,23 @@ bot.chat "Hello #{model.id} :)"
 bot.messages.select(&:assistant?).each { print "[#{_1.role}] ", _1.content, "\n" }
 ```
 
+## Reviews
+
+I supplied both Gemini and DeepSeek with the contents of [lib/](https://github.com/llmrb/llm/tree/main/lib)
+and [README.md](https://github.com/llmrb/llm#readme) via [llm-shell](https://github.com/llmrb/llm-shell#readme).
+Their feedback was way more positive than I could have imagined 😅 These are genuine responses though, with no
+special prompting or engineering. I just provided them with the source code and asked for their opinion.
+
+<details>
+  <summary>Review by Gemini</summary>
+  <img src="https://github.com/llmrb/llm/blob/main/share/llm-shell/examples/gemini.png?raw=true" alt="Gemini review" />
+</details>
+
+<details>
+  <summary>Review by DeepSeek</summary>
+  <img src="https://github.com/llmrb/llm/blob/main/share/llm-shell/examples/deepseek.png?raw=true" alt="DeepSeek review" />
+</details>
+
 ## Documentation
 
 ### API

data/lib/llm/bot.rb

@@ -123,5 +123,17 @@ module LLM
         .flat_map(&:functions)
         .select(&:pending?)
     end
+
+    ##
+    # @example
+    #   llm = LLM.openai(key: ENV["KEY"])
+    #   bot = LLM::Bot.new(llm, stream: $stdout)
+    #   bot.chat("Hello", role: :user).flush
+    # Drains the buffer and returns all messages as an array
+    # @return [Array<LLM::Message>]
+    def drain
+      messages.drain
+    end
+    alias_method :flush, :drain
   end
 end

data/lib/llm/buffer.rb

@@ -92,7 +92,8 @@ module LLM
     #   llm = LLM.openai(key: ENV["KEY"])
     #   bot = LLM::Bot.new(llm, stream: $stdout)
     #   bot.chat "Hello", role: :user
-    #   bot.messages.drain
+    #   bot.messages.flush
+    # @see LLM::Bot#drain
     # @note
     #   This method is especially useful when using the streaming API.
     # Drains the buffer and returns all messages as an array
@@ -100,6 +101,7 @@ module LLM
     def drain
       to_a
     end
+    alias_method :flush, :drain
 
     private
 
@@ -138,7 +140,7 @@ module LLM
         @response ? {previous_response_id: @response.response_id} : {}
       ].inject({}, &:merge!)
       @response = @provider.responses.create(message.content, params.merge(role:))
-      @completed.concat([*pendings, message, *@response.outputs[0]])
+      @completed.concat([*pendings, message, *@response.choices[0]])
       @pending.clear
     end
   end

data/lib/llm/client.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+module LLM
+  ##
+  # @api private
+  module Client
+    private
+
+    ##
+    # @api private
+    def persistent_client
+      mutex.synchronize do
+        if clients[client_id]
+          clients[client_id]
+        else
+          require "net/http/persistent" unless defined?(Net::HTTP::Persistent)
+          client = Net::HTTP::Persistent.new(name: self.class.name)
+          client.read_timeout = timeout
+          clients[client_id] = client
+        end
+      end
+    end
+
+    ##
+    # @api private
+    def transient_client
+      client = Net::HTTP.new(host, port)
+      client.read_timeout = timeout
+      client.use_ssl = ssl
+      client
+    end
+
+    def client_id = "#{host}:#{port}:#{timeout}:#{ssl}"
+    def clients = self.class.clients
+    def mutex = self.class.mutex
+  end
+end

data/lib/llm/function.rb

@@ -30,7 +30,7 @@
 #     fn.register(System)
 #   end
 class LLM::Function
-  class Return < Struct.new(:id, :value)
+  class Return < Struct.new(:id, :name, :value)
   end
 
   ##
@@ -92,7 +92,7 @@ class LLM::Function
   # @return [LLM::Function::Return] The result of the function call
   def call
     runner = ((Class === @runner) ? @runner.new : @runner)
-    Return.new(id, runner.call(**arguments))
+    Return.new(id, name, runner.call(**arguments))
   ensure
     @called = true
   end
@@ -106,7 +106,7 @@ class LLM::Function
   #   bot.chat bot.functions.map(&:cancel)
   # @return [LLM::Function::Return]
   def cancel(reason: "function call cancelled")
-    Return.new(id, {cancelled: true, reason:})
+    Return.new(id, name, {cancelled: true, reason:})
   ensure
     @cancelled = true
   end

data/lib/llm/message.rb

@@ -127,6 +127,16 @@ module LLM
       extra[:response]
     end
 
+    ##
+    # @note
+    #  This method might return annotations for assistant messages,
+    #  and it returns an empty array for non-assistant messages
+    # Returns annotations associated with the message
+    # @return [Array<LLM::Object>]
+    def annotations
+      @annotations ||= LLM::Object.from_hash(extra["annotations"] || [])
+    end
+
     ##
     # @note
     #  This method returns token usage for assistant messages,

data/lib/llm/object.rb

@@ -69,6 +69,12 @@ class LLM::Object < BasicObject
     to_h.dig(...)
   end
 
+  ##
+  # @return [Hash]
+  def slice(...)
+    to_h.slice(...)
+  end
+
   private
 
   def method_missing(m, *args, &b)

data/lib/llm/provider.rb

@@ -7,6 +7,19 @@
 # @abstract
 class LLM::Provider
   require "net/http"
+  require_relative "client"
+  include LLM::Client
+
+  @@clients = {}
+  @@mutex = Mutex.new
+
+  ##
+  # @api private
+  def self.clients = @@clients
+
+  ##
+  # @api private
+  def self.mutex = @@mutex
 
   ##
   # @param [String, nil] key
@@ -19,11 +32,17 @@ class LLM::Provider
   #  The number of seconds to wait for a response
   # @param [Boolean] ssl
   #  Whether to use SSL for the connection
-  def initialize(key:, host:, port: 443, timeout: 60, ssl: true)
+  # @param [Boolean] persistent
+  #  Whether to use a persistent connection.
+  #  Requires the net-http-persistent gem.
+  def initialize(key:, host:, port: 443, timeout: 60, ssl: true, persistent: false)
     @key = key
-    @client = Net::HTTP.new(host, port)
-    @client.use_ssl = ssl
-    @client.read_timeout = timeout
+    @host = host
+    @port = port
+    @timeout = timeout
+    @ssl = ssl
+    @client = persistent ? persistent_client : transient_client
+    @base_uri = URI("#{ssl ? "https" : "http"}://#{host}:#{port}/")
   end
 
   ##
@@ -217,9 +236,46 @@ class LLM::Provider
     tap { (@headers ||= {}).merge!(headers) }
   end
 
+  ##
+  # @note
+  #  This method might be outdated, and the {LLM::Provider#tool LLM::Provider#tool}
+  #  method can be used if a tool is not found here.
+  # Returns all known tools provided by a provider.
+  # @return [String => LLM::Tool]
+  def tools
+    {}
+  end
+
+  ##
+  # @note
+  #   OpenAI, Anthropic, and Gemini provide platform-tools for things
+  #   like web search, and more.
+  # Returns a tool provided by a provider.
+  # @example
+  #   llm   = LLM.openai(key: ENV["KEY"])
+  #   tools = [llm.tool(:web_search)]
+  #   res   = llm.responses.create("Summarize today's news", tools:)
+  #   print res.output_text, "\n"
+  # @param [String, Symbol] name The name of the tool
+  # @param [Hash] options Configuration options for the tool
+  # @return [LLM::Tool]
+  def tool(name, options = {})
+    LLM::Tool.new(name, options, self)
+  end
+
+  ##
+  # Provides a web search capability
+  # @param [String] query The search query
+  # @raise [NotImplementedError]
+  #  When the method is not implemented by a subclass
+  # @return [LLM::Response]
+  def web_search(query:)
+    raise NotImplementedError
+  end
+
   private
 
-  attr_reader :client
+  attr_reader :client, :base_uri, :host, :port, :timeout, :ssl
 
   ##
   # The headers to include with a request
@@ -269,8 +325,9 @@ class LLM::Provider
   #  When there is a network error at the operating system level
   # @return [Net::HTTPResponse]
   def execute(request:, stream: nil, stream_parser: self.stream_parser, &b)
+    args = (Net::HTTP === client) ? [request] : [URI.join(base_uri, request.path), request]
     res = if stream
-      client.request(request) do |res|
+      client.request(*args) do |res|
         handler = event_handler.new stream_parser.new(stream)
         parser = LLM::EventStream::Parser.new
         parser.register(handler)
@@ -284,8 +341,8 @@ class LLM::Provider
         parser&.free
       end
     else
-      b ? client.request(request) { (Net::HTTPSuccess === _1) ? b.call(_1) : _1 } :
-          client.request(request)
+      b ? client.request(*args) { (Net::HTTPSuccess === _1) ? b.call(_1) : _1 } :
+          client.request(*args)
     end
     handle_response(res)
   end

data/lib/llm/providers/anthropic/format.rb

@@ -24,7 +24,7 @@ class LLM::Anthropic
     def format_tools(params)
       return {} unless params and params[:tools]&.any?
       tools = params[:tools]
-      {tools: tools.map { _1.format(self) }}
+      {tools: tools.map { _1.respond_to?(:format) ? _1.format(self) : _1 }}
     end
   end
 end

data/lib/llm/providers/anthropic/response/web_search.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module LLM::Anthropic::Response
+  ##
+  # The {LLM::Anthropic::Response::WebSearch LLM::Anthropic::Response::WebSearch}
+  # module provides methods for accessing web search results from a web search
+  # tool call made via the {LLM::Provider#web_search LLM::Provider#web_search}
+  # method.
+  module WebSearch
+    ##
+    # Returns one or more search results
+    # @return [Array<LLM::Object>]
+    def search_results
+      LLM::Object.from_hash(
+        content
+          .select { _1["type"] == "web_search_tool_result" }
+          .flat_map { |n| n.content.map { _1.slice(:title, :url) } }
+      )
+    end
+  end
+end

data/lib/llm/providers/anthropic.rb

@@ -15,6 +15,7 @@ module LLM
   #   bot.messages.select(&:assistant?).each { print "[#{_1.role}]", _1.content, "\n" }
   class Anthropic < Provider
     require_relative "anthropic/response/completion"
+    require_relative "anthropic/response/web_search"
     require_relative "anthropic/format"
     require_relative "anthropic/error_handler"
     require_relative "anthropic/stream_parser"
@@ -83,6 +84,35 @@ module LLM
       "claude-sonnet-4-20250514"
     end
 
+    ##
+    # @note
+    #  This method includes certain tools that require configuration
+    #  through a set of options that are easier to set through the
+    #  {LLM::Provider#tool LLM::Provider#tool} method.
+    # @see https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/web-search-tool Anthropic docs
+    # @return (see LLM::Provider#tools)
+    def tools
+      {
+        bash: tool(:bash, type: "bash_20250124"),
+        web_search: tool(:web_search, type: "web_search_20250305", max_uses: 5),
+        text_editor: tool(:str_replace_based_edit_tool, type: "text_editor_20250728", max_characters: 10_000)
+      }
+    end
+
+    ##
+    # A convenience method for performing a web search using the
+    # Anthropic web search tool.
+    # @example
+    #   llm = LLM.anthropic(key: ENV["KEY"])
+    #   res = llm.web_search(query: "summarize today's news")
+    #   res.search_results.each { |item| print item.title, ": ", item.url, "\n" }
+    # @param query [String] The search query.
+    # @return [LLM::Response] The response from the LLM provider.
+    def web_search(query:)
+      complete(query, tools: [tools[:web_search]])
+        .extend(LLM::Anthropic::Response::WebSearch)
+    end
+
     private
 
     def headers

data/lib/llm/providers/gemini/format/completion_format.rb

@@ -43,7 +43,7 @@ module LLM::Gemini::Format
       when LLM::Message
         format_content(content.content)
       when LLM::Function::Return
-        [{text: JSON.dump(content.value)}]
+        [{functionResponse: {name: content.name, response: content.value}}]
       else
         prompt_error!(content)
       end

data/lib/llm/providers/gemini/format.rb

@@ -32,8 +32,9 @@ class LLM::Gemini
     # @return [Hash]
     def format_tools(params)
       return {} unless params and params[:tools]&.any?
-      functions = params.delete(:tools).grep(LLM::Function)
-      {tools: {functionDeclarations: functions.map { _1.format(self) }}}
+      tools = params.delete(:tools)
+      platform, functions = [tools.grep(LLM::Tool), tools.grep(LLM::Function)]
+      {tools: [*platform, {functionDeclarations: functions.map { _1.format(self) }}]}
     end
   end
 end

data/lib/llm/providers/gemini/images.rb

@@ -44,7 +44,7 @@ class LLM::Gemini
     def create(prompt:, model: "gemini-2.0-flash-exp-image-generation", **params)
       req  = Net::HTTP::Post.new("/v1beta/models/#{model}:generateContent?key=#{key}", headers)
       body = JSON.dump({
-        contents: [{parts: [{text: system_prompt}, {text: prompt}]}],
+        contents: [{parts: [{text: create_prompt}, {text: prompt}]}],
         generationConfig: {responseModalities: ["TEXT", "IMAGE"]}
       }.merge!(params))
       req.body = body
@@ -69,7 +69,7 @@ class LLM::Gemini
       req   = Net::HTTP::Post.new("/v1beta/models/#{model}:generateContent?key=#{key}", headers)
       image = LLM.File(image)
       body  = JSON.dump({
-        contents: [{parts: [{text: prompt}, format.format_content(image)]}],
+        contents: [{parts: [{text: edit_prompt}, {text: prompt}, format.format_content(image)]}],
         generationConfig: {responseModalities: ["TEXT", "IMAGE"]}
       }.merge!(params)).b
       set_body_stream(req, StringIO.new(body))
@@ -94,12 +94,28 @@ class LLM::Gemini
       @provider.instance_variable_get(:@key)
     end
 
-    def system_prompt
+    def create_prompt
       <<~PROMPT
-        Your task is to generate one or more image(s) from
-        text I will provide to you. Your response *MUST* include
-        at least one image, and your response *MUST NOT* include
-        any text or other content.
+        ## Context
+        Your task is to generate one or more image(s) based on the user's instructions.
+        The user will provide you with text only.
+
+        ## Instructions
+        1. The model *MUST* generate image(s) based on the user text alone.
+        2. The model *MUST NOT* generate anything else.
+      PROMPT
+    end
+
+    def edit_prompt
+      <<~PROMPT
+        ## Context
+        Your task is to edit the provided image based on the user's instructions.
+        The user will provide you with both text and an image.
+
+        ## Instructions
+        1. The model *MUST* edit the provided image based on the user's instructions
+        2. The model *MUST NOT* generate a new image.
+        3. The model *MUST NOT* generate anything else.
       PROMPT
     end
 

data/lib/llm/providers/gemini/response/completion.rb

@@ -13,8 +13,9 @@ module LLM::Gemini::Response
     def format_choices
       candidates.map.with_index do |choice, index|
         choice = LLM::Object.from_hash(choice)
-        content = choice.content
-        role, parts = content.role, content.parts
+        content = choice.content || LLM::Object.new
+        role = content.role || "model"
+        parts = content.parts || [{"text" => choice.finishReason}]
         text  = parts.filter_map { _1["text"] }.join
         tools = parts.filter_map { _1["functionCall"] }
         extra = {index:, response: self, tool_calls: format_tool_calls(tools), original_tool_calls: tools}

data/lib/llm/providers/gemini/response/web_search.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module LLM::Gemini::Response
+  ##
+  # The {LLM::Gemini::Response::WebSearch LLM::Gemini::Response::WebSearch}
+  # module provides methods for accessing web search results from a web search
+  # tool call made via the {LLM::Provider#web_search LLM::Provider#web_search}
+  # method.
+  module WebSearch
+    ##
+    # Returns one or more search results
+    # @return [Array<LLM::Object>]
+    def search_results
+      LLM::Object.from_hash(
+        candidates[0]
+          .groundingMetadata
+          .groundingChunks
+          .map { {"url" => _1.web.uri, "title" => _1.web.title} }
+      )
+    end
+  end
+end

data/lib/llm/providers/gemini/stream_parser.rb

@@ -13,7 +13,7 @@ class LLM::Gemini
     # @param [#<<] io An IO-like object
     # @return [LLM::Gemini::StreamParser]
     def initialize(io)
-      @body = LLM::Object.new
+      @body = LLM::Object.from_hash({candidates: []})
       @io = io
     end
 
@@ -21,47 +21,64 @@ class LLM::Gemini
     # @param [Hash] chunk
     # @return [LLM::Gemini::StreamParser]
     def parse!(chunk)
-      tap { merge!(chunk) }
+      tap { merge_chunk!(LLM::Object.from_hash(chunk)) }
     end
 
     private
 
-    def merge!(chunk)
+    def merge_chunk!(chunk)
       chunk.each do |key, value|
-        if key == "candidates"
-          @body.candidates ||= []
+        if key.to_s == "candidates"
           merge_candidates!(value)
+        elsif key.to_s == "usageMetadata" &&
+            @body.usageMetadata.is_a?(LLM::Object) &&
+            value.is_a?(LLM::Object)
+          @body.usageMetadata = LLM::Object.from_hash(@body.usageMetadata.to_h.merge(value.to_h))
         else
           @body[key] = value
         end
       end
     end
 
-    def merge_candidates!(candidates)
-      candidates.each.with_index do |candidate, i|
-        if @body.candidates[i].nil?
-          merge_one(@body.candidates, candidate, i)
-        else
-          merge_two(@body.candidates, candidate, i)
+    def merge_candidates!(new_candidates_list)
+      new_candidates_list.each do |new_candidate_delta|
+        index = new_candidate_delta.index
+        @body.candidates[index] ||= LLM::Object.from_hash({content: {parts: []}})
+        existing_candidate = @body.candidates[index]
+        new_candidate_delta.each do |key, value|
+          if key.to_s == "content"
+            merge_candidate_content!(existing_candidate.content, value) if value
+          else
+            existing_candidate[key] = value # Overwrite other fields
+          end
         end
       end
     end
 
-    def merge_one(candidates, candidate, i)
-      candidate
-        .dig("content", "parts")
-        &.filter_map { _1["text"] }
-        &.each { @io << _1 if @io.respond_to?(:<<) }
-      candidates[i] = candidate
+    def merge_candidate_content!(existing_content, new_content_delta)
+      new_content_delta.each do |key, value|
+        if key.to_s == "parts"
+          existing_content.parts ||= []
+          merge_content_parts!(existing_content.parts, value) if value
+        else
+          existing_content[key] = value
+        end
+      end
     end
 
-    def merge_two(candidates, candidate, i)
-      parts = candidates[i].dig("content", "parts")
-      parts&.each&.with_index do |part, j|
-        if part["text"]
-          target = candidate["content"]["parts"][j]
-          part["text"] << target["text"]
-          @io << target["text"] if @io.respond_to?(:<<)
+    def merge_content_parts!(existing_parts, new_parts_delta)
+      new_parts_delta.each do |new_part_delta|
+        if new_part_delta.text
+          last_existing_part = existing_parts.last
+          if last_existing_part&.text
+            last_existing_part.text << new_part_delta.text
+            @io << new_part_delta.text if @io.respond_to?(:<<)
+          else
+            existing_parts << new_part_delta
+            @io << new_part_delta.text if @io.respond_to?(:<<)
+          end
+        elsif new_part_delta.functionCall
+          existing_parts << new_part_delta
         end
       end
     end

data/lib/llm/providers/gemini.rb

@@ -20,6 +20,7 @@ module LLM
   class Gemini < Provider
     require_relative "gemini/response/embedding"
     require_relative "gemini/response/completion"
+    require_relative "gemini/response/web_search"
     require_relative "gemini/error_handler"
     require_relative "gemini/format"
     require_relative "gemini/stream_parser"
@@ -125,6 +126,31 @@ module LLM
       "gemini-2.5-flash"
     end
 
+    ##
+    # @note
+    #  This method includes certain tools that require configuration
+    #  through a set of options that are easier to set through the
+    #  {LLM::Provider#tool LLM::Provider#tool} method.
+    # @see https://ai.google.dev/gemini-api/docs/google-search Gemini docs
+    # @return (see LLM::Provider#tools)
+    def tools
+      {
+        google_search: tool(:google_search),
+        code_execution: tool(:code_execution),
+        url_context: tool(:url_context)
+      }
+    end
+
+    ##
+    # A convenience method for performing a web search using the
+    # Google Search tool.
+    # @param query [String] The search query.
+    # @return [LLM::Response] The response from the LLM provider.
+    def web_search(query:)
+      complete(query, tools: [tools[:google_search]])
+        .extend(LLM::Gemini::Response::WebSearch)
+    end
+
     private
 
     def headers

data/lib/llm/providers/openai/format.rb

@@ -45,7 +45,11 @@ class LLM::OpenAI
     # @return [Hash]
     def format_tools(params)
       tools = params.delete(:tools)
-      (tools.nil? || tools.empty?) ? {} : {tools: tools.map { _1.format(self) }}
+      if tools.nil? || tools.empty?
+        {}
+      else
+        {tools: tools.map { _1.respond_to?(:format) ? _1.format(self) : _1 }}
+      end
     end
   end
 end

data/lib/llm/providers/openai/response/responds.rb

@@ -2,22 +2,35 @@
 
 module LLM::OpenAI::Response
   module Responds
+    def model = body.model
     def response_id = respond_to?(:response) ? response["id"] : id
-    def outputs = [format_message]
-    def choices = body.output
-    def tools = output.select { _1.type == "function_call" }
+    def choices = [format_message]
+    def annotations = choices[0].annotations
+
+    def prompt_tokens = body.usage&.input_tokens
+    def completion_tokens = body.usage&.output_tokens
+    def total_tokens = body.usage&.total_tokens
+
+    ##
+    # Returns the aggregated text content from the response outputs.
+    # @return [String]
+    def output_text
+      choices.find(&:assistant?).content || ""
+    end
 
     private
 
     def format_message
       message = LLM::Message.new("assistant", +"", {response: self, tool_calls: []})
-      choices.each.with_index do |choice, index|
+      output.each.with_index do |choice, index|
         if choice.type == "function_call"
           message.extra[:tool_calls] << format_tool(choice)
         elsif choice.content
           choice.content.each do |c|
             next unless c["type"] == "output_text"
             message.content << c["text"] << "\n"
+            next unless c["annotations"]
+            message.extra["annotations"] = [*message.extra["annotations"], *c["annotations"]]
           end
         end
       end

data/lib/llm/providers/openai/response/web_search.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module LLM::OpenAI::Response
+  ##
+  # The {LLM::OpenAI::Response::WebSearch LLM::OpenAI::Response::WebSearch}
+  # module provides methods for accessing web search results from a web search
+  # tool call made via the {LLM::Provider#web_search LLM::Provider#web_search}
+  # method.
+  module WebSearch
+    ##
+    # Returns one or more search results
+    # @return [Array<LLM::Object>]
+    def search_results
+      LLM::Object.from_hash(
+        choices[0]
+          .annotations
+          .map { _1.slice(:title, :url) }
+      )
+    end
+  end
+end

data/lib/llm/providers/openai/stream_parser.rb

@@ -39,36 +39,44 @@ class LLM::OpenAI
     def merge_choices!(choices)
       choices.each do |choice|
         if @body.choices[choice["index"]]
-          target = @body["choices"][choice["index"]]["message"]
+          target_message = @body["choices"][choice["index"]]["message"]
           delta = choice["delta"]
           delta.each do |key, value|
-            if target[key]
-              if key == "content"
-                target[key] << value
-                @io << value if @io.respond_to?(:<<)
-              elsif key == "tool_calls"
-                merge_tools!(target, value)
-              else
-                target[key] = value
-              end
-            else
+            if key == "content"
+              target_message[key] ||= +""
+              target_message[key] << value
               @io << value if @io.respond_to?(:<<)
-              target[key] = value
+            elsif key == "tool_calls"
+              merge_tools!(target_message, value)
+            else
+              target_message[key] = value
             end
           end
         else
-          target = {"message" => {"role" => "assistant"}}
-          @body["choices"][choice["index"]] = target
-          target["message"].merge!(choice["delta"])
+          message_hash = {"role" => "assistant"}
+          @body["choices"][choice["index"]] = {"message" => message_hash}
+          choice["delta"].each do |key, value|
+            if key == "content"
+              @io << value if @io.respond_to?(:<<)
+              message_hash[key] = value
+            else
+              message_hash[key] = value
+            end
+          end
         end
       end
     end
 
     def merge_tools!(target, tools)
+      target["tool_calls"] ||= []
       tools.each.with_index do |toola, index|
         toolb = target["tool_calls"][index]
-        if toolb
-          toola["function"].each { toolb["function"][_1] << _2 }
+        if toolb && toola["function"] && toolb["function"]
+          # Append to existing function arguments
+          toola["function"].each do |func_key, func_value|
+            toolb["function"][func_key] ||= +""
+            toolb["function"][func_key] << func_value
+          end
         else
           target["tool_calls"][index] = toola
         end

data/lib/llm/providers/openai/vector_stores.rb

@@ -3,9 +3,18 @@
 class LLM::OpenAI
   ##
   # The {LLM::OpenAI::VectorStores LLM::OpenAI::VectorStores} class provides
-  # an interface for [OpenAI's vector stores API](https://platform.openai.com/docs/api-reference/vector_stores/create)
+  # an interface for [OpenAI's vector stores API](https://platform.openai.com/docs/api-reference/vector_stores/create).
+  #
+  # @example
+  #  llm = LLM.openai(key: ENV["OPENAI_SECRET"])
+  #  files = %w(foo.pdf bar.pdf).map { llm.files.create(file: _1) }
+  #  store = llm.vector_stores.create_and_poll(name: "PDF Store", file_ids: files.map(&:id))
+  #  print "[-] store is ready", "\n"
+  #  chunks = llm.vector_stores.search(vector: store, query: "What is Ruby?")
+  #  chunks.each { |chunk| puts chunk }
   class VectorStores
     require_relative "response/enumerable"
+    PollError = Class.new(LLM::Error)
 
     ##
     # @param [LLM::Provider] provider
@@ -40,6 +49,14 @@ class LLM::OpenAI
       LLM::Response.new(res)
     end
 
+    ##
+    # Create a vector store and poll until its status is "completed"
+    # @param (see LLM::OpenAI::VectorStores#create)
+    # @return (see LLM::OpenAI::VectorStores#poll)
+    def create_and_poll(...)
+      poll(vector: create(...))
+    end
+
     ##
     # Get a vector store
     # @param [String, #id] vector The ID of the vector store
@@ -181,6 +198,27 @@ class LLM::OpenAI
       LLM::Response.new(res)
     end
 
+    ##
+    # Poll a vector store until its status is "completed"
+    # @param [String, #id] vector The ID of the vector store
+    # @param [Integer] attempts The current number of attempts (default: 0)
+    # @param [Integer] max The maximum number of iterations (default: 50)
+    # @raise [LLM::PollError] When the maximum number of iterations is reached
+    # @return [LLM::Response]
+    def poll(vector:, attempts: 0, max: 50)
+      if attempts == max
+        raise LLM::PollError, "vector store '#{vector.id}' has status '#{vector.status}' after #{max} attempts"
+      elsif vector.status == "expired"
+        raise LLM::PollError, "vector store '#{vector.id}' has expired"
+      elsif vector.status != "completed"
+        vector = get(vector:)
+        sleep(0.1 * (2**attempts))
+        poll(vector:, attempts: attempts + 1, max:)
+      else
+        vector
+      end
+    end
+
     private
 
     [:headers, :execute, :set_body_stream].each do |m|

data/lib/llm/providers/openai.rb

@@ -16,6 +16,7 @@ module LLM
   class OpenAI < Provider
     require_relative "openai/response/embedding"
     require_relative "openai/response/completion"
+    require_relative "openai/response/web_search"
     require_relative "openai/error_handler"
     require_relative "openai/format"
     require_relative "openai/stream_parser"
@@ -146,6 +147,37 @@ module LLM
       "gpt-4.1"
     end
 
+    ##
+    # @note
+    #  This method includes certain tools that require configuration
+    #  through a set of options that are easier to set through the
+    #  {LLM::Provider#tool LLM::Provider#tool} method.
+    # @return (see LLM::Provider#tools)
+    def tools
+      {
+        web_search: tool(:web_search),
+        file_search: tool(:file_search),
+        image_generation: tool(:image_generation),
+        code_interpreter: tool(:code_interpreter),
+        computer_use: tool(:computer_use)
+      }
+    end
+
+    ##
+    # A convenience method for performing a web search using the
+    # OpenAI web search tool.
+    # @example
+    #   llm = LLM.openai(key: ENV["KEY"])
+    #   res = llm.web_search(query: "summarize today's news")
+    #   res.search_results.each { |item| print item.title, ": ", item.url, "\n" }
+    # @param query [String] The search query.
+    # @return [LLM::Response] The response from the LLM provider.
+    def web_search(query:)
+      responses
+        .create(query, store: false, tools: [tools[:web_search]])
+        .extend(LLM::OpenAI::Response::WebSearch)
+    end
+
     private
 
     def headers

data/lib/llm/tool.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+##
+# The {LLM::Tool LLM::Tool} class represents a platform-native tool
+# that can be activated by an LLM provider. Unlike {LLM::Function LLM::Function},
+# these tools are pre-defined by the provider and their capabilities
+# are already known to the underlying LLM.
+#
+# @example
+#   #!/usr/bin/env ruby
+#   llm = LLM.gemini ENV["KEY"]
+#   bot = LLM::Bot.new(llm, tools: [LLM.tool(:google_search)])
+#   bot.chat("Summarize today's news", role: :user)
+#   print bot.messages.find(&:assistant?).content, "\n"
+class LLM::Tool < Struct.new(:name, :options, :provider)
+  ##
+  # @return [String]
+  def to_json(...)
+    to_h.to_json(...)
+  end
+
+  ##
+  # @return [Hash]
+  def to_h
+    case provider.class.to_s
+    when "LLM::Anthropic" then options.merge("name" => name.to_s)
+    when "LLM::Gemini" then {name => options}
+    else options.merge("type" => name.to_s)
+    end
+  end
+  alias_method :to_hash, :to_h
+end

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "0.15.0"
+  VERSION = "0.16.1"
 end

data/lib/llm.rb

@@ -18,6 +18,7 @@ module LLM
   require_relative "llm/function"
   require_relative "llm/eventstream"
   require_relative "llm/eventhandler"
+  require_relative "llm/tool"
 
   module_function
 
@@ -38,7 +39,7 @@ module LLM
   end
 
   ##
-  # @param (see LLM::Provider#initialize)
+  # @param key (see LLM::Provider#initialize)
   # @return (see LLM::Ollama#initialize)
   def ollama(key: nil, **)
     require_relative "llm/providers/ollama" unless defined?(LLM::Ollama)
@@ -79,7 +80,7 @@ module LLM
   end
 
   ##
-  # Define a function
+  # Define or get a function
   # @example
   #   LLM.function(:system) do |fn|
   #     fn.description "Run system command"
@@ -94,7 +95,11 @@ module LLM
   # @param [Proc] b The block to define the function
   # @return [LLM::Function] The function object
   def function(name, &b)
-    functions[name.to_s] = LLM::Function.new(name, &b)
+    if block_given?
+      functions[name.to_s] = LLM::Function.new(name, &b)
+    else
+      functions[name.to_s]
+    end
   end
 
   ##

data/llm.gemspec

@@ -40,4 +40,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "standard", "~> 1.50"
   spec.add_development_dependency "vcr", "~> 6.0"
   spec.add_development_dependency "dotenv", "~> 2.8"
+  spec.add_development_dependency "net-http-persistent", "~> 4.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 0.15.0
+  version: 0.16.1
 platform: ruby
 authors:
 - Antar Azri
@@ -150,6 +150,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '2.8'
+- !ruby/object:Gem::Dependency
+  name: net-http-persistent
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.0'
 description: llm.rb is a zero-dependency Ruby toolkit for Large Language Models that
   includes OpenAI, Gemini, Anthropic, xAI (grok), DeepSeek, Ollama, and LlamaCpp.
   The toolkit includes full support for chat, streaming, tool calling, audio, images,
@@ -170,6 +184,7 @@ files:
 - lib/llm/bot/prompt/completion.rb
 - lib/llm/bot/prompt/respond.rb
 - lib/llm/buffer.rb
+- lib/llm/client.rb
 - lib/llm/error.rb
 - lib/llm/eventhandler.rb
 - lib/llm/eventstream.rb
@@ -193,6 +208,7 @@ files:
 - lib/llm/providers/anthropic/response/completion.rb
 - lib/llm/providers/anthropic/response/enumerable.rb
 - lib/llm/providers/anthropic/response/file.rb
+- lib/llm/providers/anthropic/response/web_search.rb
 - lib/llm/providers/anthropic/stream_parser.rb
 - lib/llm/providers/deepseek.rb
 - lib/llm/providers/deepseek/format.rb
@@ -211,6 +227,7 @@ files:
 - lib/llm/providers/gemini/response/files.rb
 - lib/llm/providers/gemini/response/image.rb
 - lib/llm/providers/gemini/response/models.rb
+- lib/llm/providers/gemini/response/web_search.rb
 - lib/llm/providers/gemini/stream_parser.rb
 - lib/llm/providers/llamacpp.rb
 - lib/llm/providers/ollama.rb
@@ -240,6 +257,7 @@ files:
 - lib/llm/providers/openai/response/image.rb
 - lib/llm/providers/openai/response/moderations.rb
 - lib/llm/providers/openai/response/responds.rb
+- lib/llm/providers/openai/response/web_search.rb
 - lib/llm/providers/openai/responses.rb
 - lib/llm/providers/openai/responses/stream_parser.rb
 - lib/llm/providers/openai/stream_parser.rb
@@ -257,6 +275,7 @@ files:
 - lib/llm/schema/object.rb
 - lib/llm/schema/string.rb
 - lib/llm/schema/version.rb
+- lib/llm/tool.rb
 - lib/llm/utils.rb
 - lib/llm/version.rb
 - llm.gemspec