llm.rb 0.6.2 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0abd0e522f099e1e53a3b4d7d3e4648a7c42d2ce23e9c17b9e09f97803e1e31d
-  data.tar.gz: 2c7b8b38570b4cafb20c61e479f16bc9a7be3e81a651a189d6e15967da796fe3
+  metadata.gz: a7175b2fe81c74e007dd41db2e0fe1bd3f3639bed375af25da0f8ed2778ea2b5
+  data.tar.gz: 1c752e61cb288fed412b342b66279e7dfdb0337705e33af3e2a1deb1d408b8d0
 SHA512:
-  metadata.gz: b309e59522e4f80a78d6a34fdf5b7aa0f06ab14ad79e845b92f5ad87537aba07532bb5b6d24bbb6d13386b32a0235ee6fc0308c7fe27496bb95da2016a27dc02
-  data.tar.gz: 61e2ecd8a53d600977f8a97592b63ed978d2659ed49513508aea2e3ab66c7fe163790eec45dc2a58ccd8aa54fca6fcb89eeb58243861c3a3238e19d86a2a00c7
+  metadata.gz: 9af91ba96e63b2c43c7f6a836db5fed48da19ba1f2bdbb48894cc71fb940eca261930fec6b8fd9a6f641fc9c69402de9cdd6fd7f9cad9a7035b69ddad04de65a
+  data.tar.gz: b3f8af44ebb2522aba58621d19a19424805471f6f9a6f8ec834ebe417e6aa08e9f0233ef19a12695e829d44ea61b164ae57697379aebdf24450b829fdc04ac25

data/README.md

@@ -2,7 +2,7 @@
 
 llm.rb is a zero-dependency Ruby toolkit for Large Language Models that
 includes OpenAI, Gemini, Anthropic, Ollama, and LlamaCpp. It’s fast, simple
-and composable – with full support for chat, tool calling, audio, 
+and composable – with full support for chat, tool calling, audio,
 images, files, and JSON Schema generation.
 
 ## Features
@@ -27,6 +27,23 @@ images, files, and JSON Schema generation.
 #### Embeddings
 - 🧮 Text embeddings and vector support
 
+## Demos
+
+<details>
+  <summary><b>1. Tools: "system" function</b></summary>
+  <img src="share/llm-shell/examples/toolcalls.gif">
+</details>
+
+<details>
+  <summary><b>2. Files: import at boot time</b></summary>
+  <img src="share/llm-shell/examples/files-boottime.gif">
+</details>
+
+<details>
+<summary><b>3. Files: import at runtime</b></summary>
+  <img src="share/llm-shell/examples/files-runtime.gif">
+</details>
+
 ## Examples
 
 ### Providers
@@ -54,6 +71,11 @@ llm = LLM.voyageai(key: "yourapikey")
 
 #### Completions
 
+> This example uses the stateless chat completions API that all
+> providers support. A similar example for OpenAI's stateful
+> responses API is available in the [docs/](docs/OPENAI_RESPONSES.md)
+> directory.
+
 The following example enables lazy mode for a
 [LLM::Chat](https://0x1eef.github.io/x/llm.rb/LLM/Chat.html)
 object by entering into a "lazy" conversation where messages are buffered and
@@ -67,13 +89,15 @@ all LLM providers support:
 #!/usr/bin/env ruby
 require "llm"
 
-llm = LLM.openai(key: ENV["KEY"])
-bot = LLM::Chat.new(llm).lazy
-bot.chat File.read("./share/llm/prompts/system.txt"), role: :system
-bot.chat "Tell me the answer to 5 + 15", role: :user
-bot.chat "Tell me the answer to (5 + 15) * 2", role: :user
-bot.chat "Tell me the answer to ((5 + 15) * 2) / 10", role: :user
-bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
+llm  = LLM.openai(key: ENV["KEY"])
+bot  = LLM::Chat.new(llm).lazy
+msgs = bot.chat do |prompt|
+  prompt.system File.read("./share/llm/prompts/system.txt")
+  prompt.user "Tell me the answer to 5 + 15"
+  prompt.user "Tell me the answer to (5 + 15) * 2"
+  prompt.user "Tell me the answer to ((5 + 15) * 2) / 10"
+end
+msgs.each { print "[#{_1.role}] ", _1.content, "\n" }
 
 ##
 # [system] You are my math assistant.
@@ -91,46 +115,6 @@ bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
 #             The answer to ((5 + 15) * 2) / 10 is 4.
 ```
 
-#### Responses
-
-The responses API is a recent addition
-[provided by OpenAI](https://platform.openai.com/docs/guides/conversation-state?api-mode=responses)
-that lets a client store message state on their servers &ndash; and in turn
-a client can avoid maintaining state manually as well as avoid sending
-the entire conversation with each request that is made. Although it is
-primarily supported by OpenAI at the moment, we might see other providers
-support it in the future. For now
-[llm.rb supports the responses API](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Responses.html)
-for the OpenAI provider:
-
-```ruby
-#!/usr/bin/env ruby
-require "llm"
-
-llm = LLM.openai(key: ENV["KEY"])
-bot = LLM::Chat.new(llm).lazy
-bot.respond File.read("./share/llm/prompts/system.txt"), role: :developer
-bot.respond "Tell me the answer to 5 + 15", role: :user
-bot.respond "Tell me the answer to (5 + 15) * 2", role: :user
-bot.respond "Tell me the answer to ((5 + 15) * 2) / 10", role: :user
-bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
-
-##
-# [developer] You are my math assistant.
-#             I will provide you with (simple) equations.
-#             You will provide answers in the format "The answer to <equation> is <answer>".
-#             I will provide you a set of messages. Reply to all of them.
-#             A message is considered unanswered if there is no corresponding assistant response.
-#
-# [user] Tell me the answer to 5 + 15
-# [user] Tell me the answer to (5 + 15) * 2
-# [user] Tell me the answer to ((5 + 15) * 2) / 10
-#
-# [assistant] The answer to 5 + 15 is 20.
-#             The answer to (5 + 15) * 2 is 40.
-#             The answer to ((5 + 15) * 2) / 10 is 4.
-```
-
 ### Schema
 
 #### Structured
@@ -139,13 +123,9 @@ All LLM providers except Anthropic allow a client to describe the structure
 of a response that a LLM emits according to a schema that is described by JSON.
 The schema lets a client describe what JSON object (or value) an LLM should emit,
 and the LLM will abide by the schema. See also: [JSON Schema website](https://json-schema.org/overview/what-is-jsonschema).
-
-True to the llm.rb spirit of doing one thing well, and solving problems through the
-composition of objects, the generation of a schema is delegated to another object
-who is responsible for and an expert in the generation of JSON schemas. We will use
-the
+ We will use the
 [llmrb/json-schema](https://github.com/llmrb/json-schema)
-library for the sake of the examples &ndash; the interface is designed so you 
+library for the sake of the examples &ndash; the interface is designed so you
 could drop in any other library in its place:
 
 ```ruby
@@ -153,11 +133,11 @@ could drop in any other library in its place:
 require "llm"
 
 llm = LLM.openai(key: ENV["KEY"])
-schema = llm.schema.object({os: llm.schema.string.enum("OpenBSD", "FreeBSD", "NetBSD")})
+schema = llm.schema.object({fruit: llm.schema.string.enum("Apple", "Orange", "Pineapple")})
 bot = LLM::Chat.new(llm, schema:)
-bot.chat "You secretly love NetBSD", role: :system
-bot.chat "What operating system is the best?", role: :user
-bot.messages.find(&:assistant?).content! # => {os: "NetBSD"}
+bot.chat "Your favorite fruit is Pineapple", role: :system
+bot.chat "What fruit is your favorite?", role: :user
+bot.messages.find(&:assistant?).content! # => {fruit: "Pineapple"}
 
 schema = llm.schema.object({answer: llm.schema.integer.required})
 bot = LLM::Chat.new(llm, schema:)
@@ -228,8 +208,7 @@ Some but not all providers implement audio generation capabilities that
 can create speech from text, transcribe audio to text, or translate
 audio to text (usually English). The following example uses the OpenAI provider
 to create an audio file from a text prompt. The audio is then moved to
-`${HOME}/hello.mp3` as the final step. As always, consult the provider's
-documentation for more information on how to use the audio generation API:
+`${HOME}/hello.mp3` as the final step:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -245,8 +224,7 @@ IO.copy_stream res.audio, File.join(Dir.home, "hello.mp3")
 The following example transcribes an audio file to text. The audio file
 (`${HOME}/hello.mp3`) was theoretically created in the previous example,
 and the result is printed to the console. The example uses the OpenAI
-provider to transcribe the audio file. As always, consult the provider's
-documentation for more information on how to use the audio transcription API:
+provider to transcribe the audio file:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -264,9 +242,7 @@ print res.text, "\n" # => "Hello world."
 The following example translates an audio file to text. In this example
 the audio file (`${HOME}/bomdia.mp3`) is theoretically in Portuguese,
 and it is translated to English. The example uses the OpenAI provider,
-and at the time of writing, it can only translate to English. As always,
-consult the provider's documentation for more information on how to use
-the audio translation API:
+and at the time of writing, it can only translate to English:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -308,11 +284,7 @@ end
 The following example is focused on editing a local image with the aid
 of a prompt. The image (`/images/cat.png`) is returned to us with the cat
 now wearing a hat. The image is then moved to `${HOME}/catwithhat.png` as
-the final step.
-
-Results and quality may vary, consider prompt adjustments if the results
-are not as expected, and consult the provider's documentation
-for more information on how to use the image editing API:
+the final step:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -336,8 +308,7 @@ end
 The following example is focused on creating variations of a local image.
 The image (`/images/cat.png`) is returned to us with five different variations.
 The images are then moved to `${HOME}/catvariation0.png`, `${HOME}/catvariation1.png`
-and so on as the final step. Consult the provider's documentation for more information
-on how to use the image variations API:
+and so on as the final step:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -458,10 +429,8 @@ print res.embeddings[0].size, "\n"
 Almost all LLM providers provide a models endpoint that allows a client to
 query the list of models that are available to use. The list is dynamic,
 maintained by LLM providers, and it is independent of a specific llm.rb release.
-True to the llm.rb spirit of small, composable objects that cooperate with
-each other, a
 [LLM::Model](https://0x1eef.github.io/x/llm.rb/LLM/Model.html)
-object can be used instead of a string that describes a model name (although
+objects can be used instead of a string that describes a model name (although
 either works). Let's take a look at an example:
 
 ```ruby
@@ -497,7 +466,8 @@ over or doesn't cover at all. The API reference is available at
 
 The [docs/](docs/) directory contains some additional documentation that
 didn't quite make it into the README. It covers the design guidelines that
-the library follows, and some strategies for memory management.
+the library follows, some strategies for memory management, and other
+provider-specific features.
 
 ## See also
 
@@ -506,7 +476,7 @@ the library follows, and some strategies for memory management.
 An extensible, developer-oriented command line utility that is powered by
 llm.rb and serves as a demonstration of the library's capabilities. The
 [demo](https://github.com/llmrb/llm-shell#demos) section has a number of GIF
-previews might be especially interesting!
+previews might be especially interesting.
 
 ## Install
 

data/lib/llm/chat/builder.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+class LLM::Chat
+  ##
+  # @private
+  module Builder
+    private
+
+    def create_response!(prompt, params)
+      @provider.responses.create(
+        prompt,
+        @params.merge(params.merge(@response ? {previous_response_id: @response.id} : {}))
+      )
+    end
+
+    def create_completion!(prompt, params)
+      @provider.complete(
+        prompt,
+        @params.merge(params.merge(messages:))
+      )
+    end
+  end
+end

data/lib/llm/chat/conversable.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+class LLM::Chat
+  ##
+  # @private
+  module Conversable
+    private
+
+    def async_response(prompt, params = {})
+      role = params.delete(:role)
+      @messages << [LLM::Message.new(role, prompt), @params.merge(params), :respond]
+    end
+
+    def sync_response(prompt, params = {})
+      role = params[:role]
+      @response = create_response!(prompt, params)
+      @messages.concat [Message.new(role, prompt), @response.outputs[0]]
+    end
+
+    def async_completion(prompt, params = {})
+      role = params.delete(:role)
+      @messages.push [LLM::Message.new(role, prompt), @params.merge(params), :complete]
+    end
+
+    def sync_completion(prompt, params = {})
+      role = params[:role]
+      completion = create_completion!(prompt, params)
+      @messages.concat [Message.new(role, prompt), completion.choices[0]]
+    end
+
+    include LLM
+  end
+end

data/lib/llm/chat/prompt/completion.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module LLM::Chat::Prompt
+  class Completion < Struct.new(:bot)
+    ##
+    # @param [String] prompt
+    # @param [Hash] params (see LLM::Provider#complete)
+    # @return [LLM::Chat]
+    def system(prompt, params = {})
+      bot.chat prompt, params.merge(role: :system)
+    end
+
+    ##
+    # @param [String] prompt
+    # @param [Hash] params (see LLM::Provider#complete)
+    # @return [LLM::Chat]
+    def user(prompt, params = {})
+      bot.chat prompt, params.merge(role: :user)
+    end
+  end
+end

data/lib/llm/chat/prompt/respond.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module LLM::Chat::Prompt
+  class Respond < Struct.new(:bot)
+    ##
+    # @param [String] prompt
+    # @param [Hash] params (see LLM::Provider#complete)
+    # @return [LLM::Chat]
+    def system(prompt, params = {})
+      bot.respond prompt, params.merge(role: :system)
+    end
+
+    ##
+    # @param [String] prompt
+    # @param [Hash] params (see LLM::Provider#complete)
+    # @return [LLM::Chat]
+    def developer(prompt, params = {})
+      bot.respond prompt, params.merge(role: :developer)
+    end
+
+    ##
+    # @param [String] prompt
+    # @param [Hash] params (see LLM::Provider#complete)
+    # @return [LLM::Chat]
+    def user(prompt, params = {})
+      bot.respond prompt, params.merge(role: :user)
+    end
+  end
+end

data/lib/llm/chat.rb

@@ -11,14 +11,36 @@ module LLM
   #   #!/usr/bin/env ruby
   #   require "llm"
   #
+  #   llm  = LLM.openai(ENV["KEY"])
+  #   bot  = LLM::Chat.new(llm).lazy
+  #   msgs = bot.chat do |prompt|
+  #     prompt.system "Answer the following questions."
+  #     prompt.user "What is 5 + 7 ?"
+  #     prompt.user "Why is the sky blue ?"
+  #     prompt.user "Why did the chicken cross the road ?"
+  #   end
+  #   msgs.map { print "[#{_1.role}]", _1.content, "\n" }
+  #
+  # @example
+  #   #!/usr/bin/env ruby
+  #   require "llm"
+  #
   #   llm = LLM.openai(ENV["KEY"])
   #   bot = LLM::Chat.new(llm).lazy
-  #   bot.chat("Provide short and concise answers", role: :system)
-  #   bot.chat("What is 5 + 7 ?", role: :user)
-  #   bot.chat("Why is the sky blue ?", role: :user)
-  #   bot.chat("Why did the chicken cross the road ?", role: :user)
+  #   bot.chat "Answer the following questions.", role: :system
+  #   bot.chat "What is 5 + 7 ?", role: :user
+  #   bot.chat "Why is the sky blue ?", role: :user
+  #   bot.chat "Why did the chicken cross the road ?", role: :user
   #   bot.messages.map { print "[#{_1.role}]", _1.content, "\n" }
   class Chat
+    require_relative "chat/prompt/completion"
+    require_relative "chat/prompt/respond"
+    require_relative "chat/conversable"
+    require_relative "chat/builder"
+
+    include Conversable
+    include Builder
+
     ##
     # @return [Array<LLM::Message>]
     attr_reader :messages
@@ -44,18 +66,18 @@ module LLM
     # Maintain a conversation via the chat completions API
     # @param prompt (see LLM::Provider#complete)
     # @param params (see LLM::Provider#complete)
-    # @return [LLM::Chat]
-    def chat(prompt, params = {})
-      params = {role: :user}.merge!(params)
-      if lazy?
-        role = params.delete(:role)
-        @messages << [LLM::Message.new(role, prompt), @params.merge(params), :complete]
-        self
+    # @yieldparam [LLM::Chat::CompletionPrompt] prompt Yields a prompt
+    # @return [LLM::Chat, Array<LLM::Message>, LLM::Buffer]
+    #  Returns self unless given a block, otherwise returns messages
+    def chat(prompt = nil, params = {})
+      if block_given?
+        yield Prompt::Completion.new(self)
+        messages
+      elsif prompt.nil?
+        raise ArgumentError, "wrong number of arguments (given 0, expected 1)"
       else
-        role = params[:role]
-        completion = complete!(prompt, params)
-        @messages.concat [Message.new(role, prompt), completion.choices[0]]
-        self
+        params = {role: :user}.merge!(params)
+        tap { lazy? ? async_completion(prompt, params) : sync_completion(prompt, params) }
       end
     end
 
@@ -64,36 +86,20 @@ module LLM
     # @note Not all LLM providers support this API
     # @param prompt (see LLM::Provider#complete)
     # @param params (see LLM::Provider#complete)
-    # @return [LLM::Chat]
-    def respond(prompt, params = {})
-      params = {role: :user}.merge!(params)
-      if lazy?
-        role = params.delete(:role)
-        @messages << [LLM::Message.new(role, prompt), @params.merge(params), :respond]
-        self
+    # @return [LLM::Chat, Array<LLM::Message>, LLM::Buffer]
+    #  Returns self unless given a block, otherwise returns messages
+    def respond(prompt = nil, params = {})
+      if block_given?
+        yield Prompt::Respond.new(self)
+        messages
+      elsif prompt.nil?
+        raise ArgumentError, "wrong number of arguments (given 0, expected 1)"
       else
-        role = params[:role]
-        @response = respond!(prompt, params)
-        @messages.concat [Message.new(role, prompt), @response.outputs[0]]
-        self
+        params = {role: :user}.merge!(params)
+        tap { lazy? ? async_response(prompt, params) : sync_response(prompt, params) }
       end
     end
 
-    ##
-    # The last message in the conversation.
-    # @note
-    #  The `read_response` and `recent_message` methods are aliases of
-    #  the `last_message` method, and you can choose the name that best
-    #  fits your context or code style.
-    # @param [#to_s] role
-    #  The role of the last message.
-    # @return [LLM::Message]
-    def last_message(role: @provider.assistant_role)
-      messages.reverse_each.find { _1.role == role.to_s }
-    end
-    alias_method :recent_message, :last_message
-    alias_method :read_response, :last_message
-
     ##
     # Enables lazy mode for the conversation.
     # @return [LLM::Chat]
@@ -121,13 +127,13 @@ module LLM
     end
 
     ##
-    # Returns an array of functions that have yet to be called
+    # Returns an array of functions that can be called
     # @return [Array<LLM::Function>]
     def functions
       messages
         .select(&:assistant?)
         .flat_map(&:functions)
-        .reject(&:called?)
+        .select(&:pending?)
     end
 
     private
@@ -144,19 +150,5 @@ module LLM
       end
     end
     private_constant :Array
-
-    def respond!(prompt, params)
-      @provider.responses.create(
-        prompt,
-        @params.merge(params.merge(@response ? {previous_response_id: @response.id} : {}))
-      )
-    end
-
-    def complete!(prompt, params)
-      @provider.complete(
-        prompt,
-        @params.merge(params.merge(messages:))
-      )
-    end
   end
 end

data/lib/llm/function.rb

@@ -1,5 +1,37 @@
 # frozen_string_literal: true
 
+##
+# The {LLM::Function LLM::Function} class represents a function that can
+# be called by an LLM. It comes in two forms: a Proc-based function,
+# or a Class-based function.
+#
+# @example
+#   # Proc-based
+#   LLM.function(:system) do |fn|
+#     fn.description "Runs system commands, emits their output"
+#     fn.params do |schema|
+#       schema.object(command: schema.string.required)
+#     end
+#     fn.define do |params|
+#       Kernel.system(params.command)
+#     end
+#   end
+#
+# @example
+#   # Class-based
+#   class System
+#     def call(params)
+#       Kernel.system(params.command)
+#     end
+#   end
+#
+#   LLM.function(:system) do |fn|
+#     fn.description "Runs system commands, emits their output"
+#     fn.params do |schema|
+#       schema.object(command: schema.string.required)
+#     end
+#     fn.register(System)
+#   end
 class LLM::Function
   class Return < Struct.new(:id, :value)
   end
@@ -25,6 +57,8 @@ class LLM::Function
   def initialize(name, &b)
     @name = name
     @schema = JSON::Schema.new
+    @called = false
+    @cancelled = false
     yield(self)
   end
 
@@ -45,21 +79,36 @@ class LLM::Function
 
   ##
   # Set the function implementation
-  # @param [Proc] b The function implementation
+  # @param [Proc, Class] b The function implementation
   # @return [void]
-  def define(&b)
-    @runner = b
+  def define(klass = nil, &b)
+    @runner = klass || b
   end
+  alias_method :register, :define
 
   ##
   # Call the function
-  # @return [Object] The result of the function call
+  # @return [LLM::Function::Return] The result of the function call
   def call
-    Return.new id, @runner.call(arguments)
+    Return.new id, (Class === @runner) ? @runner.new.call(arguments) : @runner.call(arguments)
   ensure
     @called = true
   end
 
+  ##
+  # Returns a value that communicates that the function call was cancelled
+  # @example
+  #   llm = LLM.openai(key: ENV["KEY"])
+  #   bot = LLM::Chat.new(llm, tools: [fn1, fn2])
+  #   bot.chat "I want to run the functions"
+  #   bot.chat bot.functions.map(&:cancel)
+  # @return [LLM::Function::Return]
+  def cancel(reason: "function call cancelled")
+    Return.new(id, {cancelled: true, reason:})
+  ensure
+    @cancelled = true
+  end
+
   ##
   # Returns true when a function has been called
   # @return [Boolean]
@@ -67,6 +116,20 @@ class LLM::Function
     @called
   end
 
+  ##
+  # Returns true when a function has been cancelled
+  # @return [Boolean]
+  def cancelled?
+    @cancelled
+  end
+
+  ##
+  # Returns true when a function has neither been called nor cancelled
+  # @return [Boolean]
+  def pending?
+    !@called && !@cancelled
+  end
+
   ##
   # @return [Hash]
   def format(provider)

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "0.6.2"
+  VERSION = "0.7.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 0.6.2
+  version: 0.7.0
 platform: ruby
 authors:
 - Antar Azri
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2025-05-07 00:00:00.000000000 Z
+date: 2025-05-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: webmock
@@ -176,6 +176,10 @@ files:
 - lib/llm.rb
 - lib/llm/buffer.rb
 - lib/llm/chat.rb
+- lib/llm/chat/builder.rb
+- lib/llm/chat/conversable.rb
+- lib/llm/chat/prompt/completion.rb
+- lib/llm/chat/prompt/respond.rb
 - lib/llm/core_ext/ostruct.rb
 - lib/llm/error.rb
 - lib/llm/file.rb