llm.rb 0.8.0 → 0.9.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9b4e83ac151c51faaa4a1e275058091a9ce6f61c3dc10e879a6215b0f1498aad
-  data.tar.gz: f78b7bbeaece69384d6b38014e9d1d99816195d8536a310a25d2a23479dda122
+  metadata.gz: 20812e4c8cfc7ebee81190054e7483b00b24e0f0f567f630bfb4dc0ac962193f
+  data.tar.gz: 4ff26fa74520b29da3b6aa10331d5cd618e13d4df53cb9f0c5b7a7691f5fb42e
 SHA512:
-  metadata.gz: e117602fae5643713a159d633201cd88e94a339763710bbb788b3b1439e39bbbff9f2c221975fc58e1b57aabdf8d0d935d69dbc6acbece84e98701e129cf3c3d
-  data.tar.gz: 79f2ef053bf500ba9e5ab76c62abdb69ab93ba43b2f11fce867d008e64fbe09e154c1a30fbfdeb08ae30a5442b5d7e5876aa42c788dce8d0778c786f0a69adee
+  metadata.gz: 4ed6e3f0426fc0967cb59cc14ae0d0afe552a93e2a29b8d173bf000341b77800ea31ef9088d45a696d353ac0e7f58cc1b3d7f86c3c15d757ae73efb33523b56d
+  data.tar.gz: 136e14863ef92264e270f3b6616a9660298cdc477a2a15db1b89b9360be1ce2be3dde0ee66d97be68d57c5bf68f4f0fc9764657187fbf2ae226f1a5ed6579189

data/README.md

@@ -1,21 +1,22 @@
 ## About
 
 llm.rb is a zero-dependency Ruby toolkit for Large Language Models that
-includes OpenAI, Gemini, Anthropic, DeepSeek, Ollama, and LlamaCpp.
-It's fast, simple and composable – with full support for chat,
-tool calling, audio, images, files, and JSON Schema generation.
+includes OpenAI, Gemini, Anthropic, DeepSeek, Ollama, and LlamaCpp. The
+toolkit includes full support for chat, streaming, tool calling, audio,
+images, files, and JSON Schema generation.
 
 ## Features
 
 #### General
 - ✅ A single unified interface for multiple providers
 - 📦 Zero dependencies outside Ruby's standard library
-- 🚀 Optimized for performance and low memory usage
+- 🚀 Efficient API design that minimizes the request count
 
 #### Chat, Agents
 - 🧠 Stateless and stateful chat via completions and responses API
 - 🤖 Tool calling and function execution
 - 🗂️ JSON Schema support for structured, validated responses
+- 📡 Streaming support for real-time response updates
 
 #### Media
 - 🗣️ Text-to-speech, transcription, and translation
@@ -31,17 +32,17 @@ tool calling, audio, images, files, and JSON Schema generation.
 
 <details>
   <summary><b>1. Tools: "system" function</b></summary>
-  <img src="share/llm-shell/examples/toolcalls.gif">
+  <img src="https://github.com/llmrb/llm/raw/main/share/llm-shell/examples/toolcalls.gif">
 </details>
 
 <details>
   <summary><b>2. Files: import at runtime</b></summary>
-  <img src="share/llm-shell/examples/files-runtime.gif">
+  <img src="https://github.com/llmrb/llm/raw/main/share/llm-shell/examples/files-runtime.gif">
 </details>
 
 <details>
   <summary><b>3. Files: import at boot time</b></summary>
-  <img src="share/llm-shell/examples/files-boottime.gif">
+  <img src="https://github.com/llmrb/llm/raw/main/share/llm-shell/examples/files-boottime.gif">
 </details>
 
 ## Examples
@@ -60,7 +61,7 @@ using an API key (if required) and an optional set of configuration options via
 require "llm"
 
 ##
-# cloud providers
+# remote providers
 llm = LLM.openai(key: "yourapikey")
 llm = LLM.gemini(key: "yourapikey")
 llm = LLM.anthropic(key: "yourapikey")
@@ -79,24 +80,24 @@ llm = LLM.llamacpp(key: nil)
 
 > This example uses the stateless chat completions API that all
 > providers support. A similar example for OpenAI's stateful
-> responses API is available in the [docs/](docs/OPENAI.md)
+> responses API is available in the [docs/](docs/OPENAI.md#responses)
 > directory.
 
-The following example enables lazy mode for a
-[LLM::Chat](https://0x1eef.github.io/x/llm.rb/LLM/Chat.html)
-object by entering into a conversation where messages are buffered and
-sent to the provider only when necessary. Both lazy and non-lazy conversations
-maintain a message thread that can be reused as context throughout a conversation.
-The example captures the spirit of llm.rb by demonstrating how objects cooperate
-together through composition, and it uses the stateless chat completions API that
-all LLM providers support:
+The following example creates an instance of
+[LLM::Bot](https://0x1eef.github.io/x/llm.rb/LLM/Bot.html)
+by entering into a conversation where messages are buffered and
+sent to the provider on-demand. This is the default behavior
+because it can reduce the number of requests sent to a provider,
+and avoids unneccessary requests until an attempt to iterate over
+[LLM::Bot#messages](https://0x1eef.github.io/x/llm.rb/LLM/Bot.html#messages-instance_method)
+is made:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm  = LLM.openai(key: ENV["KEY"])
-bot  = LLM::Chat.new(llm).lazy
+bot  = LLM::Bot.new(llm)
 msgs = bot.chat do |prompt|
   prompt.system File.read("./share/llm/prompts/system.txt")
   prompt.user "Tell me the answer to 5 + 15"
@@ -106,21 +107,38 @@ end
 
 # At this point, we execute a single request
 msgs.each { print "[#{_1.role}] ", _1.content, "\n" }
+```
 
-##
-# [system] You are my math assistant.
-#          I will provide you with (simple) equations.
-#          You will provide answers in the format "The answer to <equation> is <answer>".
-#          I will provide you a set of messages. Reply to all of them.
-#          A message is considered unanswered if there is no corresponding assistant response.
-#
-# [user] Tell me the answer to 5 + 15
-# [user] Tell me the answer to (5 + 15) * 2
-# [user] Tell me the answer to ((5 + 15) * 2) / 10
-#
-# [assistant] The answer to 5 + 15 is 20.
-#             The answer to (5 + 15) * 2 is 40.
-#             The answer to ((5 + 15) * 2) / 10 is 4.
+#### Streaming
+
+> There Is More Than One Way To Do It (TIMTOWTDI) when you are
+> using llm.rb &ndash; and this is especially true when it
+> comes to streaming. See the streaming documentation in
+> [docs/](docs/STREAMING.md#flexibility) for more details.
+
+The following example streams the messages in a conversation
+as they are generated in real-time. This feature can be useful
+in case you want to see the contents of a message as it is
+generated, or in case you want to avoid potential read timeouts
+during the generation of a response.
+
+The `stream` option can be set to an IO object, or the value `true`
+to enable streaming &ndash; and at the end of the request, `bot.chat`
+returns the same response as the non-streaming version which allows
+you to process a response in the same way:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+bot = LLM::Bot.new(llm)
+bot.chat(stream: $stdout) do |prompt|
+  prompt.system "You are my math assistant."
+  prompt.user "Tell me the answer to 5 + 15"
+  prompt.user "Tell me the answer to (5 + 15) * 2"
+  prompt.user "Tell me the answer to ((5 + 15) * 2) / 10"
+end.to_a
 ```
 
 ### Schema
@@ -130,12 +148,7 @@ msgs.each { print "[#{_1.role}] ", _1.content, "\n" }
 All LLM providers except Anthropic and DeepSeek allow a client to describe
 the structure of a response that a LLM emits according to a schema that is
 described by JSON. The schema lets a client describe what JSON object (or value)
-an LLM should emit, and the LLM will abide by the schema.
-See also: [JSON Schema website](https://json-schema.org/overview/what-is-jsonschema).
-We will use the
-[llmrb/json-schema](https://github.com/llmrb/json-schema)
-library for the sake of the examples &ndash; the interface is designed so you
-could drop in any other library in its place:
+an LLM should emit, and the LLM will abide by the schema:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -145,14 +158,14 @@ require "llm"
 # Objects
 llm = LLM.openai(key: ENV["KEY"])
 schema = llm.schema.object(answer: llm.schema.integer.required)
-bot = LLM::Chat.new(llm, schema:).lazy
+bot = LLM::Bot.new(llm, schema:)
 bot.chat "Does the earth orbit the sun?", role: :user
 bot.messages.find(&:assistant?).content! # => {probability: 1}
 
 ##
 # Enums
 schema = llm.schema.object(fruit: llm.schema.string.enum("Apple", "Orange", "Pineapple"))
-bot = LLM::Chat.new(llm, schema:).lazy
+bot = LLM::Bot.new(llm, schema:)
 bot.chat "Your favorite fruit is Pineapple", role: :system
 bot.chat "What fruit is your favorite?", role: :user
 bot.messages.find(&:assistant?).content! # => {fruit: "Pineapple"}
@@ -160,7 +173,7 @@ bot.messages.find(&:assistant?).content! # => {fruit: "Pineapple"}
 ##
 # Arrays
 schema = llm.schema.object(answers: llm.schema.array(llm.schema.integer.required))
-bot = LLM::Chat.new(llm, schema:).lazy
+bot = LLM::Bot.new(llm, schema:)
 bot.chat "Answer all of my questions", role: :system
 bot.chat "Tell me the answer to ((5 + 5) / 2)", role: :user
 bot.chat "Tell me the answer to ((5 + 5) / 2) * 2", role: :user
@@ -172,14 +185,14 @@ bot.messages.find(&:assistant?).content! # => {answers: [5, 10, 11]}
 
 #### Functions
 
-The OpenAI, Anthropic, Gemini and Ollama providers support a powerful feature known as
-tool calling, and although it is a little complex to understand at first,
-it can be powerful for building agents. The following example demonstrates how we
-can define a local function (which happens to be a tool), and OpenAI can
-then detect when we should call the function.
+All providers support a powerful feature known as tool calling, and although
+it is a little complex to understand at first, it can be powerful for building
+agents. The following example demonstrates how we can define a local function
+(which happens to be a tool), and a provider (such as OpenAI) can then detect
+when we should call the function.
 
 The
-[LLM::Chat#functions](https://0x1eef.github.io/x/llm.rb/LLM/Chat.html#functions-instance_method)
+[LLM::Bot#functions](https://0x1eef.github.io/x/llm.rb/LLM/Bot.html#functions-instance_method)
 method returns an array of functions that can be called after sending a message and
 it will only be populated if the LLM detects a function should be called. Each function
 corresponds to an element in the "tools" array. The array is emptied after a function call,
@@ -208,7 +221,7 @@ tool = LLM.function(:system) do |fn|
   end
 end
 
-bot = LLM::Chat.new(llm, tools: [tool]).lazy
+bot = LLM::Bot.new(llm, tools: [tool])
 bot.chat "Your task is to run shell commands via a tool.", role: :system
 
 bot.chat "What is the current date?", role: :user
@@ -367,7 +380,7 @@ can be given to the chat method:
 require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-bot = LLM::Chat.new(llm).lazy
+bot = LLM::Bot.new(llm)
 file = llm.files.create(file: "/documents/openbsd_is_awesome.pdf")
 bot.chat(file)
 bot.chat("What is this file about?")
@@ -398,7 +411,7 @@ to a prompt:
 require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-bot = LLM::Chat.new(llm).lazy
+bot = LLM::Bot.new(llm)
 
 bot.chat [URI("https://example.com/path/to/image.png"), "Describe the image in the link"]
 bot.messages.select(&:assistant?).each { print "[#{_1.role}] ", _1.content, "\n" }
@@ -469,7 +482,7 @@ end
 ##
 # Select a model
 model = llm.models.all.find { |m| m.id == "gpt-3.5-turbo" }
-bot = LLM::Chat.new(llm, model:)
+bot = LLM::Bot.new(llm, model:)
 bot.chat "Hello #{model.id} :)"
 bot.messages.select(&:assistant?).each { print "[#{_1.role}] ", _1.content, "\n" }
 ```

data/lib/llm/{chat → bot}/builder.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-class LLM::Chat
+class LLM::Bot
   ##
   # @private
   module Builder

data/lib/llm/bot/conversable.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+class LLM::Bot
+  ##
+  # @private
+  module Conversable
+    private
+
+    ##
+    # Queues a response to be sent to the provider.
+    # @param [String] prompt The prompt
+    # @param [Hash] params
+    # @return [void]
+    def async_response(prompt, params = {})
+      role = params.delete(:role)
+      @messages << [LLM::Message.new(role, prompt), @params.merge(params), :respond]
+    end
+
+    ##
+    # Queues a completion to be sent to the provider.
+    # @param [String] prompt The prompt
+    # @param [Hash] params
+    # @return [void]
+    def async_completion(prompt, params = {})
+      role = params.delete(:role)
+      @messages.push [LLM::Message.new(role, prompt), @params.merge(params), :complete]
+    end
+  end
+end

data/lib/llm/{chat → bot}/prompt/completion.rb

@@ -1,20 +1,30 @@
 # frozen_string_literal: true
 
-module LLM::Chat::Prompt
-  class Completion < Struct.new(:bot)
+module LLM::Bot::Prompt
+  class Completion < Struct.new(:bot, :defaults)
+    ##
+    # @param [LLM::Bot] bot
+    # @param [Hash] defaults
+    # @return [LLM::Bot::Prompt::Completion]
+    def initialize(bot, defaults)
+      super(bot, defaults || {})
+    end
+
     ##
     # @param [String] prompt
     # @param [Hash] params (see LLM::Provider#complete)
-    # @return [LLM::Chat]
+    # @return [LLM::Bot]
     def system(prompt, params = {})
+      params = defaults.merge(params)
       bot.chat prompt, params.merge(role: :system)
     end
 
     ##
     # @param [String] prompt
     # @param [Hash] params (see LLM::Provider#complete)
-    # @return [LLM::Chat]
+    # @return [LLM::Bot]
     def user(prompt, params = {})
+      params = defaults.merge(params)
       bot.chat prompt, params.merge(role: :user)
     end
   end

data/lib/llm/{chat → bot}/prompt/respond.rb

@@ -1,28 +1,39 @@
 # frozen_string_literal: true
 
-module LLM::Chat::Prompt
-  class Respond < Struct.new(:bot)
+module LLM::Bot::Prompt
+  class Respond < Struct.new(:bot, :defaults)
+    ##
+    # @param [LLM::Bot] bot
+    # @param [Hash] defaults
+    # @return [LLM::Bot::Prompt::Completion]
+    def initialize(bot, defaults)
+      super(bot, defaults || {})
+    end
+
     ##
     # @param [String] prompt
     # @param [Hash] params (see LLM::Provider#complete)
-    # @return [LLM::Chat]
+    # @return [LLM::Bot]
     def system(prompt, params = {})
+      params = defaults.merge(params)
       bot.respond prompt, params.merge(role: :system)
     end
 
     ##
     # @param [String] prompt
     # @param [Hash] params (see LLM::Provider#complete)
-    # @return [LLM::Chat]
+    # @return [LLM::Bot]
     def developer(prompt, params = {})
+      params = defaults.merge(params)
       bot.respond prompt, params.merge(role: :developer)
     end
 
     ##
     # @param [String] prompt
     # @param [Hash] params (see LLM::Provider#complete)
-    # @return [LLM::Chat]
+    # @return [LLM::Bot]
     def user(prompt, params = {})
+      params = defaults.merge(params)
       bot.respond prompt, params.merge(role: :user)
     end
   end

data/lib/llm/{chat.rb → bot.rb}

@@ -2,47 +2,48 @@
 
 module LLM
   ##
-  # {LLM::Chat LLM::Chat} provides a chat object that maintains a
-  # thread of messages that acts as context throughout a conversation.
-  # A conversation can use the chat completions API that most LLM providers
-  # support or the responses API that a select few LLM providers support.
+  # {LLM::Bot LLM::Bot} provides a bot object that can maintain a
+  # a conversation. A conversation can use the chat completions API
+  # that all LLM providers support or the responses API that a select
+  # few LLM providers support.
   #
-  # @example
+  # @example example #1
   #   #!/usr/bin/env ruby
   #   require "llm"
   #
   #   llm  = LLM.openai(ENV["KEY"])
-  #   bot  = LLM::Chat.new(llm).lazy
+  #   bot  = LLM::Bot.new(llm)
   #   msgs = bot.chat do |prompt|
   #     prompt.system "Answer the following questions."
   #     prompt.user "What is 5 + 7 ?"
   #     prompt.user "Why is the sky blue ?"
   #     prompt.user "Why did the chicken cross the road ?"
   #   end
-  #   msgs.map { print "[#{_1.role}]", _1.content, "\n" }
+  #   msgs.each { print "[#{_1.role}]", _1.content, "\n" }
   #
-  # @example
+  # @example example #2
   #   #!/usr/bin/env ruby
   #   require "llm"
   #
   #   llm = LLM.openai(ENV["KEY"])
-  #   bot = LLM::Chat.new(llm).lazy
+  #   bot = LLM::Bot.new(llm)
   #   bot.chat "Answer the following questions.", role: :system
   #   bot.chat "What is 5 + 7 ?", role: :user
   #   bot.chat "Why is the sky blue ?", role: :user
   #   bot.chat "Why did the chicken cross the road ?", role: :user
-  #   bot.messages.map { print "[#{_1.role}]", _1.content, "\n" }
-  class Chat
-    require_relative "chat/prompt/completion"
-    require_relative "chat/prompt/respond"
-    require_relative "chat/conversable"
-    require_relative "chat/builder"
+  #   bot.messages.each { print "[#{_1.role}]", _1.content, "\n" }
+  class Bot
+    require_relative "bot/prompt/completion"
+    require_relative "bot/prompt/respond"
+    require_relative "bot/conversable"
+    require_relative "bot/builder"
 
     include Conversable
     include Builder
 
     ##
-    # @return [Array<LLM::Message>]
+    # Returns an Enumerable for the messages in a conversation
+    # @return [LLM::Buffer<LLM::Message>]
     attr_reader :messages
 
     ##
@@ -58,72 +59,68 @@ module LLM
     def initialize(provider, params = {})
       @provider = provider
       @params = {model: provider.default_model, schema: nil}.compact.merge!(params)
-      @lazy = false
-      @messages = [].extend(Array)
+      @messages = LLM::Buffer.new(provider)
     end
 
     ##
     # Maintain a conversation via the chat completions API
-    # @param prompt (see LLM::Provider#complete)
-    # @param params (see LLM::Provider#complete)
-    # @yieldparam [LLM::Chat::CompletionPrompt] prompt Yields a prompt
-    # @return [LLM::Chat, Array<LLM::Message>, LLM::Buffer]
-    #  Returns self unless given a block, otherwise returns messages
+    # @overload def chat(prompt, params = {})
+    #   @param prompt (see LLM::Provider#complete)
+    #   @param params The params
+    #   @return [LLM::Bot]
+    #     Returns self
+    # @overload def chat(prompt, params, &block)
+    #   @param prompt (see LLM::Provider#complete)
+    #   @param params The params
+    #   @yield prompt Yields a prompt
+    #   @return [LLM::Buffer]
+    #     Returns messages
     def chat(prompt = nil, params = {})
       if block_given?
-        yield Prompt::Completion.new(self)
+        params = prompt
+        yield Prompt::Completion.new(self, params)
         messages
       elsif prompt.nil?
         raise ArgumentError, "wrong number of arguments (given 0, expected 1)"
       else
         params = {role: :user}.merge!(params)
-        tap { lazy? ? async_completion(prompt, params) : sync_completion(prompt, params) }
+        tap { async_completion(prompt, params) }
       end
     end
 
     ##
     # Maintain a conversation via the responses API
-    # @note Not all LLM providers support this API
-    # @param prompt (see LLM::Provider#complete)
-    # @param params (see LLM::Provider#complete)
-    # @return [LLM::Chat, Array<LLM::Message>, LLM::Buffer]
-    #  Returns self unless given a block, otherwise returns messages
+    # @overload def respond(prompt, params = {})
+    #   @param prompt (see LLM::Provider#complete)
+    #   @param params The params
+    #   @return [LLM::Bot]
+    #     Returns self
+    # @overload def respond(prompt, params, &block)
+    #   @note Not all LLM providers support this API
+    #   @param prompt (see LLM::Provider#complete)
+    #   @param params The params
+    #   @yield prompt Yields a prompt
+    #   @return [LLM::Buffer]
+    #     Returns messages
     def respond(prompt = nil, params = {})
       if block_given?
-        yield Prompt::Respond.new(self)
+        params = prompt
+        yield Prompt::Respond.new(self, params)
         messages
       elsif prompt.nil?
         raise ArgumentError, "wrong number of arguments (given 0, expected 1)"
       else
         params = {role: :user}.merge!(params)
-        tap { lazy? ? async_response(prompt, params) : sync_response(prompt, params) }
+        tap { async_response(prompt, params) }
       end
     end
 
-    ##
-    # Enables lazy mode for the conversation.
-    # @return [LLM::Chat]
-    def lazy
-      tap do
-        next if lazy?
-        @lazy = true
-        @messages = LLM::Buffer.new(@provider)
-      end
-    end
-
-    ##
-    # @return [Boolean]
-    #  Returns true if the conversation is lazy
-    def lazy?
-      @lazy
-    end
-
     ##
     # @return [String]
     def inspect
       "#<#{self.class.name}:0x#{object_id.to_s(16)} " \
       "@provider=#{@provider.class}, @params=#{@params.inspect}, " \
-      "@messages=#{@messages.inspect}, @lazy=#{@lazy.inspect}>"
+      "@messages=#{@messages.inspect}>"
     end
 
     ##
@@ -135,20 +132,5 @@ module LLM
         .flat_map(&:functions)
         .select(&:pending?)
     end
-
-    private
-
-    ##
-    # @private
-    module Array
-      def find(...)
-        reverse_each.find(...)
-      end
-
-      def unread
-        reject(&:read?)
-      end
-    end
-    private_constant :Array
   end
 end

data/lib/llm/buffer.rb

@@ -23,9 +23,13 @@ module LLM
     #  Yields each message in the conversation thread
     # @raise (see LLM::Provider#complete)
     # @return [void]
-    def each
-      empty! unless @pending.empty?
-      @completed.each { yield(_1) }
+    def each(...)
+      if block_given?
+        empty! unless @pending.empty?
+        @completed.each { yield(_1) }
+      else
+        enum_for(:each, ...)
+      end
     end
 
     ##

data/lib/llm/error.rb

@@ -8,34 +8,34 @@ module LLM
       block_given? ? yield(self) : nil
       super
     end
+  end
 
+  ##
+  # The superclass of all HTTP protocol errors
+  class ResponseError < Error
     ##
-    # The superclass of all HTTP protocol errors
-    class ResponseError < Error
-      ##
-      # @return [Net::HTTPResponse]
-      #  Returns the response associated with an error
-      attr_accessor :response
+    # @return [Net::HTTPResponse]
+    #  Returns the response associated with an error
+    attr_accessor :response
 
-      def message
-        [super, response.body].join("\n")
-      end
+    def message
+      [super, response.body].join("\n")
     end
+  end
 
-    ##
-    # HTTPUnauthorized
-    Unauthorized = Class.new(ResponseError)
+  ##
+  # HTTPUnauthorized
+  UnauthorizedError = Class.new(ResponseError)
 
-    ##
-    # HTTPTooManyRequests
-    RateLimit = Class.new(ResponseError)
+  ##
+  # HTTPTooManyRequests
+  RateLimitError = Class.new(ResponseError)
 
-    ##
-    # When an given an input that is not understood
-    FormatError = Class.new(Error)
+  ##
+  # When an given an input object that is not understood
+  FormatError = Class.new(Error)
 
-    ##
-    # When given a prompt that is not understood
-    PromptError = Class.new(FormatError)
-  end
+  ##
+  # When given a prompt object that is not understood
+  PromptError = Class.new(FormatError)
 end