llm.rb 0.2.1 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 732a483717a5ec8e443077fb71294b1e301c3a8867b225c1fc2a58bd02fe3130
-  data.tar.gz: a1c2591a07c413cebfdffa99d133855bb177cc4a6607860333dbc9991da8d33e
+  metadata.gz: 9073b7495fb9bdad2deec1d2c086b6d3b554c5a440dd884108a2fa8d12f7c8a9
+  data.tar.gz: 514902fc97de61dc18df8c22d51d9e86472a62e1ffb0c4ce4394b0684cddbd8a
 SHA512:
-  metadata.gz: 4f5983f97b3c1e25f4147ec81f6d91df5073ea03dc4031690979f68b2053bf73bae07f7c57c3f7c9813dfd5b43eb1bd7364d5f5929234013d6b19bb49f9271ec
-  data.tar.gz: 286f560ce2d9e048e481796d27fd6c9a658b92cec0267f9527fad49bf5349ece0d05b943c086cce3c12bbd82f53dbf086ed91c516d39b815bd6106edca21914a
+  metadata.gz: 0d0c35fa38ed3481872e29131d15e03e5a4bf0ad8a96c42ba64a5f48ed32584973d39b53ca630c966d54b6700a83a44abb1f4224c1bb9c1ca7f9e7a2d953e1c3
+  data.tar.gz: 8889034558c56a2bc1ff5321cf0ca45d82ac83ac7122c741e859caed7d060b34b99824cc53d20a5add4949dd135cf65c383f8400e7c112a44110fc1d4e0d2f4d

data/README.md

@@ -2,7 +2,8 @@
 
 llm.rb is a lightweight library that provides a common interface
 and set of functionality for multiple Large Language Models (LLMs). It
-is designed to be simple, flexible, and easy to use.
+is designed to be simple, flexible, and easy to use – and it has been
+implemented with no dependencies outside Ruby's standard library.
 
 ## Examples
 
@@ -10,10 +11,10 @@ is designed to be simple, flexible, and easy to use.
 
 #### LLM::Provider
 
-All providers inherit from [LLM::Provider](https://0x1eef.github.io/x/llm/LLM/Provider.html) –
+All providers inherit from [LLM::Provider](https://0x1eef.github.io/x/llm.rb/LLM/Provider.html) –
 they share a common interface and set of functionality. Each provider can be instantiated
 using an API key (if required) and an optional set of configuration options via
-[the singleton methods of LLM](https://0x1eef.github.io/x/llm/LLM.html). For example:
+[the singleton methods of LLM](https://0x1eef.github.io/x/llm.rb/LLM.html). For example:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -25,37 +26,28 @@ llm = LLM.anthropic("yourapikey")
 llm = LLM.ollama(nil)
 ```
 
-### Completions
+### Conversations
 
-#### Conversation
+#### Completions
 
-The
-[LLM::Provider#chat](https://0x1eef.github.io/x/llm/LLM/Provider.html#chat-instance_method)
-method returns a lazy-variant of a
-[LLM::Conversation](https://0x1eef.github.io/x/llm/LLM/Conversation.html)
-object, and it allows for a "lazy" conversation where messages are batched and
-sent to the provider only when necessary. The non-lazy counterpart is available via the
-[LLM::Provider#chat!](https://0x1eef.github.io/x/llm/LLM/Provider.html#chat!-instance_method)
-method.
-
-Both lazy and non-lazy conversations maintain a message thread that can
-be reused as context throughout a conversation. For the sake of brevity the system
-prompt is loaded from
-[a file](./share/llm/prompts/system.txt)
-in the following example – all other prompts are "user" prompts –
-and a single request is made to the provider when iterating over the messages
-belonging to a lazy conversation:
+The following example enables lazy mode for a
+[LLM::Chat](https://0x1eef.github.io/x/llm.rb/LLM/Chat.html)
+object by entering into a "lazy" conversation where messages are buffered and
+sent to the provider only when necessary.  Both lazy and non-lazy conversations
+maintain a message thread that can be reused as context throughout a conversation.
+The example uses the stateless chat completions API that all LLM providers support:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm = LLM.openai(ENV["KEY"])
-convo = llm.chat File.read("./share/llm/prompts/system.txt"), :system
-convo.chat "Tell me the answer to 5 + 15"
-convo.chat "Tell me the answer to (5 + 15) * 2"
-convo.chat "Tell me the answer to ((5 + 15) * 2) / 10"
-convo.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
+bot = LLM::Chat.new(llm).lazy
+bot.chat File.read("./share/llm/prompts/system.txt"), :system
+bot.chat "Tell me the answer to 5 + 15", :user
+bot.chat "Tell me the answer to (5 + 15) * 2", :user
+bot.chat "Tell me the answer to ((5 + 15) * 2) / 10", :user
+bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
 
 ##
 # [system] You are my math assistant.
@@ -73,128 +65,288 @@ convo.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
 #             The answer to ((5 + 15) * 2) / 10 is 4.
 ```
 
-#### Prompts
-
-Both lazy and non-lazy conversations accept text as a prompt.
-Depending on the provider, they may also accept a
-[URI](https://docs.ruby-lang.org/en/master/URI.html)
-or
-[LLM::File](https://0x1eef.github.io/x/llm/LLM/File.html)
-object. Generally a
-[URI](https://docs.ruby-lang.org/en/master/URI.html)
-object is used to reference an image on the web, and an
-[LLM::File](https://0x1eef.github.io/x/llm/LLM/File.html)
-object is used to reference a file on the local filesystem.
-The following list shows the types of prompts that each
-provider accepts:
-
-* OpenAI       =>   String, URI
-* Gemini        =>   String, LLM::File
-* Anthropic   =>   String, URI
-* Ollama        =>   String, URI
+#### Responses
+
+The responses API is a recent addition
+[provided by OpenAI](https://platform.openai.com/docs/guides/conversation-state?api-mode=responses)
+that lets a client store message state on their servers – and in turn
+a client can avoid maintaining state manually as well as avoid sending
+the entire conversation with each request that is made. Although it is
+primarily supported by OpenAI at the moment, we might see other providers
+support it in the future. For now
+[llm.rb supports the responses API](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Responses.html)
+for the OpenAI provider:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(ENV["KEY"])
+bot = LLM::Chat.new(llm).lazy
+bot.respond File.read("./share/llm/prompts/system.txt"), :developer
+bot.respond "Tell me the answer to 5 + 15", :user
+bot.respond "Tell me the answer to (5 + 15) * 2", :user
+bot.respond "Tell me the answer to ((5 + 15) * 2) / 10", :user
+bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
+
+##
+# [developer] You are my math assistant.
+#             I will provide you with (simple) equations.
+#             You will provide answers in the format "The answer to <equation> is <answer>".
+#             I will provide you a set of messages. Reply to all of them.
+#             A message is considered unanswered if there is no corresponding assistant response.
+#
+# [user] Tell me the answer to 5 + 15
+# [user] Tell me the answer to (5 + 15) * 2
+# [user] Tell me the answer to ((5 + 15) * 2) / 10
+#
+# [assistant] The answer to 5 + 15 is 20.
+#             The answer to (5 + 15) * 2 is 40.
+#             The answer to ((5 + 15) * 2) / 10 is 4.
+```
+
+### Audio
+
+#### Speech
+
+Some but not all providers implement audio generation capabilities that
+can create text from speech, transcribe audio to text, or translate
+audio to text (usually English). The following example uses the OpenAI provider
+to create an audio file from a text prompt. The audio is then moved to
+`${HOME}/hello.mp3` as the final step. As always, consult the provider's
+documentation (eg [OpenAI docs](https://platform.openai.com/docs/api-reference/audio/create))
+for more information on how to use the audio generation API:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.audio.create_speech(input: "Hello world")
+File.binwrite File.join(Dir.home, "hello.mp3"),
+	          res.audio.string
+```
+
+#### Transcribe
+
+The following example transcribes an audio file to text. The audio file
+(`${HOME}/hello.mp3`) was theoretically created in the previous example,
+and the result is printed to the console. The example uses the OpenAI
+provider to transcribe the audio file. As always, consult the provider's
+documentation (eg
+[OpenAI docs](https://platform.openai.com/docs/api-reference/audio/createTranscription),
+[Gemini docs](https://ai.google.dev/gemini-api/docs/audio))
+for more information on how to use the audio transcription API.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Audio](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Audio.html),
+[LLM::OpenAI::Audio](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Audio.html)):
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.audio.create_transcription(
+  file: LLM::File(File.join(Dir.home, "hello.mp3"))
+)
+print res.text, "\n" # => "Hello world."
+```
+
+#### Translate
+
+The following example translates an audio file to text. In this example
+the audio file (`${HOME}/bomdia.mp3`) is theoretically in Portuguese,
+and it is translated to English. The example uses the OpenAI provider,
+and at the time of writing, it can only translate to English. As always,
+consult the provider's documentation (eg
+[OpenAI docs](https://platform.openai.com/docs/api-reference/audio/createTranslation),
+[Gemini docs](https://ai.google.dev/gemini-api/docs/audio))
+for more information on how to use the audio translation API.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Audio](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Audio.html),
+[LLM::OpenAI::Audio](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Audio.html)):
+
+
+```ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.audio.create_translation(
+  file: LLM::File(File.join(Dir.home, "bomdia.mp3"))
+)
+print res.text, "\n" # => "Good morning."
+```
+
+### Images
+
+#### Create
+
+Some but all LLM providers implement image generation capabilities that
+can create new images from a prompt, or edit an existing image with a
+prompt. The following example uses the OpenAI provider to create an
+image of a dog on a rocket to the moon. The image is then moved to
+`${HOME}/dogonrocket.png` as the final step.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Images](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Images.html),
+[LLM::OpenAI::Images](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Images.html)):
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.images.create(prompt: "a dog on a rocket to the moon")
+res.urls.each do |url|
+  FileUtils.mv OpenURI.open_uri(url).path,
+               File.join(Dir.home, "dogonrocket.png")
+end
+```
+
+#### Edit
+
+The following example is focused on editing a local image with the aid
+of a prompt. The image (`/images/cat.png`) is returned to us with the cat
+now wearing a hat. The image is then moved to `${HOME}/catwithhat.png` as
+the final step.
+
+Results and quality may vary, consider prompt adjustments if the results
+are not satisfactory, and consult the provider's documentation
+(eg
+[OpenAI docs](https://platform.openai.com/docs/api-reference/images/createEdit),
+[Gemini docs](https://ai.google.dev/gemini-api/docs/image-generation))
+for more information on how to use the image editing API.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Images](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Images.html),
+[LLM::OpenAI::Images](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Images.html)):
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.images.edit(
+  image: LLM::File("/images/cat.png"),
+  prompt: "a cat with a hat",
+)
+res.urls.each do |url|
+  FileUtils.mv OpenURI.open_uri(url).path,
+               File.join(Dir.home, "catwithhat.png")
+end
+```
+
+#### Variations
+
+The following example is focused on creating variations of a local image.
+The image (`/images/cat.png`) is returned to us with five different variations.
+The images are then moved to `${HOME}/catvariation0.png`, `${HOME}/catvariation1.png`
+and so on as the final step. Consult the provider's documentation
+(eg [OpenAI docs](https://platform.openai.com/docs/api-reference/images/createVariation))
+for more information on how to use the image variations API:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.images.create_variation(
+  image: LLM::File("/images/cat.png"),
+  n: 5
+)
+res.urls.each.with_index do |url, index|
+  FileUtils.mv OpenURI.open_uri(url).path,
+               File.join(Dir.home, "catvariation#{index}.png")
+end
+```
 
 ### Embeddings
 
 #### Text
 
 The
-[`LLM::Provider#embed`](https://0x1eef.github.io/x/llm/LLM/Provider.html#embed-instance_method)
+[`LLM::Provider#embed`](https://0x1eef.github.io/x/llm.rb/LLM/Provider.html#embed-instance_method)
 method generates a vector representation of one or more chunks
 of text. Embeddings capture the semantic meaning of text &ndash;
 a common use-case for them is to store chunks of text in a
 vector database, and then to query the database for *semantically
 similar* text. These chunks of similar text can then support the
 generation of a prompt that is used to query a large language model,
-which will go on to generate a response.
-
-For example, a user query might find similar text that adds important
-context to the prompt that informs the large language model in how to respond.
-The chunks of text may also carry metadata that can be used to further filter
-and contextualize the search results. This technique is popularly known as
-retrieval-augmented generation (RAG). Embeddings can also be used for
-other purposes as well &ndash; RAG is just one of the most popular use-cases.
-
-Let's take a look at an example that generates a couple of vectors
-for two chunks of text:
+which will go on to generate a response:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm = LLM.openai(ENV["KEY"])
-res = llm.embed(["programming is fun", "ruby is a programming language"])
+res = llm.embed(["programming is fun", "ruby is a programming language", "sushi is art"])
 print res.class, "\n"
 print res.embeddings.size, "\n"
 print res.embeddings[0].size, "\n"
 
 ##
 # LLM::Response::Embedding
-# 2
+# 3
 # 1536
 ```
 
-### LLM
+### Memory
 
-#### Timeouts
+#### Child process
 
-When running the ollama provider locally it might take a while for
-the language model to reply &ndash; depending on hardware and the
-size of the model. The following example demonstrates how to wait
-a longer period of time for a response through the use of the
-`timeout` configuration option with the `qwq` model. The following
-example waits up to 15 minutes for a response:
+When it comes to the generation of audio, images, and video memory consumption
+can be a potential problem. There are a few strategies in place to deal with this,
+and one lesser known strategy is to let a child process handle the memory cost
+by delegating media generation to a child process.
 
-```ruby
-#!/usr/bin/env ruby
-require "llm"
-
-llm = LLM.ollama(nil, timeout: 60*15)
-llm.chat "What is the meaning of life ?", model: "qwq"
-llm.last_message.tap { print "[assistant] ", _1.content, "\n" }
-```
-
-#### Models
-
-Generally each Large Language Model provides multiple models to choose
-from, and each model has its own set of capabilities and limitations.
-The following example demonstrates how to query the list of models
-through the
-[LLM::Provider#models](http://0x1eef.github.io/x/llm/LLM/Provider.html#models-instance_method)
-method &ndash; the example happens to use the ollama provider but
-this can be done for any provider:
+Once a child process exits, any memory it had used is freed immediately and
+the parent process can continue to have a small memory footprint. In a sense
+it is similar to being able to use malloc + free from Ruby. The following example
+demonstrates how that might look like in practice:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
-##
-# List models
-llm = LLM.ollama(nil)
-llm.models.each { print "#{_2.name}: #{_2.description}", "\n" }
-
-##
-# Select a model
-llm.chat "Hello, world!", model: llm.models["qwq"]
-
-##
-# This also works
-llm.chat "Hello, world!", model: "qwq"
+llm = LLM.gemini(ENV["KEY"])
+fork do
+  %w[dog cat sheep goat capybara].each do |animal|
+    res = llm.images.create(prompt: "a #{animal} on a rocket to the moon")
+    File.binwrite "#{animal}.png", res.images[0].binary
+  end
+end
+Process.wait
 ```
-## Providers
 
-- [x] [Anthropic](https://www.anthropic.com/)
-- [x] [OpenAI](https://platform.openai.com/docs/overview)
-- [x] [Gemini](https://ai.google.dev/gemini-api/docs)
-- [x] [Ollama](https://github.com/ollama/ollama#readme)
-- [ ] Hugging Face
-- [ ] Cohere
-- [ ] AI21 Labs
-- [ ] Replicate
-- [ ] Mistral AI
+## API reference
 
-## Documentation
+The README tries to provide a high-level overview of the library. For everything
+else there's the API reference. It covers classes and methods that the README glances
+over or doesn't cover at all. The API reference is available at
+[0x1eef.github.io/x/llm.rb](https://0x1eef.github.io/x/llm.rb).
 
-A complete API reference is available at [0x1eef.github.io/x/llm](https://0x1eef.github.io/x/llm)
 
 ## Install
 
@@ -204,4 +356,6 @@ llm.rb can be installed via rubygems.org:
 
 ## License
 
-MIT. See [LICENSE.txt](LICENSE.txt) for more details
+[BSD Zero Clause](https://choosealicense.com/licenses/0bsd/)
+<br>
+See [LICENSE](./LICENSE)

data/lib/llm/buffer.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+module LLM
+  ##
+  # @private
+  # {LLM::Buffer LLM::Buffer} provides an Enumerable object that
+  # yields each message in a conversation on-demand, and only sends
+  # a request to the LLM when a response is needed.
+  class Buffer
+    include Enumerable
+
+    ##
+    # @param [LLM::Provider] provider
+    # @return [LLM::Buffer]
+    def initialize(provider)
+      @provider = provider
+      @pending = []
+      @completed = []
+    end
+
+    ##
+    # @yield [LLM::Message]
+    #  Yields each message in the conversation thread
+    # @raise (see LLM::Provider#complete)
+    # @return [void]
+    def each
+      empty! unless @pending.empty?
+      @completed.each { yield(_1) }
+    end
+
+    ##
+    # @param [[LLM::Message, Hash]] item
+    #  A message and its parameters
+    # @return [void]
+    def <<(item)
+      @pending << item
+      self
+    end
+    alias_method :push, :<<
+
+    ##
+    # @return [String]
+    def inspect
+      "#<#{self.class.name}:0x#{object_id.to_s(16)} " \
+      "completed_count=#{@completed.size} pending_count=#{@pending.size}>"
+    end
+
+    private
+
+    def empty!
+      message, params, method = @pending[-1]
+      if method == :complete
+        complete!(message, params)
+      elsif method == :respond
+        respond!(message, params)
+      else
+        raise LLM::Error, "Unknown method: #{method}"
+      end
+    end
+
+    def complete!(message, params)
+      messages = @pending[0..-2].map { _1[0] }
+      completion = @provider.complete(
+        message.content,
+        message.role,
+        **params.merge(messages:)
+      )
+      @completed.concat([*messages, message, completion.choices[0]])
+      @pending.clear
+    end
+
+    def respond!(message, params)
+      input = @pending[0..-2].map { _1[0] }
+      @response = @provider.responses.create(
+        message.content,
+        message.role,
+        **params.merge(input:).merge(@response ? {previous_response_id: @response.id} : {})
+      )
+      @completed.concat([*input, message, @response.outputs[0]])
+      @pending.clear
+    end
+  end
+end

data/lib/llm/chat.rb

@@ -0,0 +1,131 @@
+# frozen_string_literal: true
+
+module LLM
+  ##
+  # {LLM::Chat LLM::Chat} provides a chat object that maintains a
+  # thread of messages that acts as context throughout a conversation.
+  # A conversation can use the chat completions API that most LLM providers
+  # support or the responses API that a select few LLM providers support.
+  #
+  # @example
+  #   #!/usr/bin/env ruby
+  #   require "llm"
+  #
+  #   llm = LLM.openai(ENV["KEY"])
+  #   bot = LLM::Chat.new(llm).lazy
+  #   bot.chat("Your task is to answer all of my questions", :system)
+  #   bot.chat("Your answers should be short and concise", :system)
+  #   bot.chat("What is 5 + 7 ?", :user)
+  #   bot.chat("Why is the sky blue ?", :user)
+  #   bot.chat("Why did the chicken cross the road ?", :user)
+  #   bot.messages.map { print "[#{_1.role}]", _1.content, "\n" }
+  class Chat
+    ##
+    # @return [Array<LLM::Message>]
+    attr_reader :messages
+
+    ##
+    # @param [LLM::Provider] provider
+    #  A provider
+    # @param [Hash] params
+    #  The parameters to maintain throughout the conversation
+    def initialize(provider, params = {})
+      @provider = provider
+      @params = params
+      @lazy = false
+      @messages = []
+    end
+
+    ##
+    # Maintain a conversation via the chat completions API
+    # @param prompt (see LLM::Provider#prompt)
+    # @param role (see LLM::Provider#prompt)
+    # @param params (see LLM::Provider#prompt)
+    # @return [LLM::Chat]
+    def chat(prompt, role = :user, **params)
+      if lazy?
+        @messages << [LLM::Message.new(role, prompt), @params.merge(params), :complete]
+        self
+      else
+        completion = complete!(prompt, role, params)
+        @messages.concat [Message.new(role, prompt), completion.choices[0]]
+        self
+      end
+    end
+
+    ##
+    # Maintain a conversation via the responses API
+    # @note Not all LLM providers support this API
+    # @param prompt (see LLM::Provider#prompt)
+    # @param role (see LLM::Provider#prompt)
+    # @param params (see LLM::Provider#prompt)
+    # @return [LLM::Chat]
+    def respond(prompt, role = :user, **params)
+      if lazy?
+        @messages << [LLM::Message.new(role, prompt), @params.merge(params), :respond]
+        self
+      else
+        @response = respond!(prompt, role, params)
+        @messages.concat [Message.new(role, prompt), @response.outputs[0]]
+        self
+      end
+    end
+
+    ##
+    # The last message in the conversation.
+    # @note
+    #  The `read_response` and `recent_message` methods are aliases of
+    #  the `last_message` method, and you can choose the name that best
+    #  fits your context or code style.
+    # @param [#to_s] role
+    #  The role of the last message.
+    # @return [LLM::Message]
+    def last_message(role: @provider.assistant_role)
+      messages.reverse_each.find { _1.role == role.to_s }
+    end
+    alias_method :recent_message, :last_message
+    alias_method :read_response, :last_message
+
+    ##
+    # Enables lazy mode for the conversation.
+    # @return [LLM::Chat]
+    def lazy
+      tap do
+        next if lazy?
+        @lazy = true
+        @messages = LLM::Buffer.new(@provider)
+      end
+    end
+
+    ##
+    # @return [Boolean]
+    #  Returns true if the conversation is lazy
+    def lazy?
+      @lazy
+    end
+
+    def inspect
+      "#<#{self.class.name}:0x#{object_id.to_s(16)} " \
+      "@provider=#{@provider.class}, @params=#{@params.inspect}, " \
+      "@messages=#{@messages.inspect}, @lazy=#{@lazy.inspect}>"
+    end
+
+    private
+
+    def respond!(prompt, role, params)
+      @provider.responses.create(
+        prompt,
+        role,
+        **@params.merge(params.merge(@response ? {previous_response_id: @response.id} : {}))
+      )
+    end
+
+    def complete!(prompt, role, params)
+      @provider.complete(
+        prompt,
+        role,
+        **@params.merge(params.merge(messages:))
+      )
+    end
+  end
+end