llm.rb 0.2.1 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 732a483717a5ec8e443077fb71294b1e301c3a8867b225c1fc2a58bd02fe3130
-  data.tar.gz: a1c2591a07c413cebfdffa99d133855bb177cc4a6607860333dbc9991da8d33e
+  metadata.gz: 3939075c064b4abfd8853c3f67b6db7df6111d340d658d4d8ad0c4d1bccc96bc
+  data.tar.gz: 0ca274d3e4b032c25730aef896df903681c28033ebb0907c965339a33aff56d1
 SHA512:
-  metadata.gz: 4f5983f97b3c1e25f4147ec81f6d91df5073ea03dc4031690979f68b2053bf73bae07f7c57c3f7c9813dfd5b43eb1bd7364d5f5929234013d6b19bb49f9271ec
-  data.tar.gz: 286f560ce2d9e048e481796d27fd6c9a658b92cec0267f9527fad49bf5349ece0d05b943c086cce3c12bbd82f53dbf086ed91c516d39b815bd6106edca21914a
+  metadata.gz: feaf87457b8fa5b4f756a5fe8cc1f670c8b0286a730fe00273bc99678092fe7f704d58f01ba0a0baf4072a0dcee063bc87cf88bc7cdf53125334476adbce41f6
+  data.tar.gz: 3be8b460d9b483c0e172d9159b2394ea39da7a1475aee3ab47b224303e2a251f3b04f0543402494485040998225f84342be986db8c7b8ea80df92f561d4d6d92

data/README.md

@@ -2,7 +2,10 @@
 
 llm.rb is a lightweight library that provides a common interface
 and set of functionality for multiple Large Language Models (LLMs). It
-is designed to be simple, flexible, and easy to use.
+is designed to be simple, flexible, and easy to use – and it has been
+implemented with zero dependencies outside Ruby's standard library. See the
+[philosophy](#philosophy) section for more information on the design principles
+behind llm.rb.
 
 ## Examples
 
@@ -10,10 +13,10 @@ is designed to be simple, flexible, and easy to use.
 
 #### LLM::Provider
 
-All providers inherit from [LLM::Provider](https://0x1eef.github.io/x/llm/LLM/Provider.html) –
+All providers inherit from [LLM::Provider](https://0x1eef.github.io/x/llm.rb/LLM/Provider.html) –
 they share a common interface and set of functionality. Each provider can be instantiated
 using an API key (if required) and an optional set of configuration options via
-[the singleton methods of LLM](https://0x1eef.github.io/x/llm/LLM.html). For example:
+[the singleton methods of LLM](https://0x1eef.github.io/x/llm.rb/LLM.html). For example:
 
 ```ruby
 #!/usr/bin/env ruby
@@ -25,37 +28,28 @@ llm = LLM.anthropic("yourapikey")
 llm = LLM.ollama(nil)
 ```
 
-### Completions
+### Conversations
 
-#### Conversation
+#### Completions
 
-The
-[LLM::Provider#chat](https://0x1eef.github.io/x/llm/LLM/Provider.html#chat-instance_method)
-method returns a lazy-variant of a
-[LLM::Conversation](https://0x1eef.github.io/x/llm/LLM/Conversation.html)
-object, and it allows for a "lazy" conversation where messages are batched and
-sent to the provider only when necessary. The non-lazy counterpart is available via the
-[LLM::Provider#chat!](https://0x1eef.github.io/x/llm/LLM/Provider.html#chat!-instance_method)
-method.
-
-Both lazy and non-lazy conversations maintain a message thread that can
-be reused as context throughout a conversation. For the sake of brevity the system
-prompt is loaded from
-[a file](./share/llm/prompts/system.txt)
-in the following example – all other prompts are "user" prompts –
-and a single request is made to the provider when iterating over the messages
-belonging to a lazy conversation:
+The following example enables lazy mode for a
+[LLM::Chat](https://0x1eef.github.io/x/llm.rb/LLM/Chat.html)
+object by entering into a "lazy" conversation where messages are buffered and
+sent to the provider only when necessary.  Both lazy and non-lazy conversations
+maintain a message thread that can be reused as context throughout a conversation.
+The example uses the stateless chat completions API that all LLM providers support:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm = LLM.openai(ENV["KEY"])
-convo = llm.chat File.read("./share/llm/prompts/system.txt"), :system
-convo.chat "Tell me the answer to 5 + 15"
-convo.chat "Tell me the answer to (5 + 15) * 2"
-convo.chat "Tell me the answer to ((5 + 15) * 2) / 10"
-convo.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
+bot = LLM::Chat.new(llm).lazy
+bot.chat File.read("./share/llm/prompts/system.txt"), :system
+bot.chat "Tell me the answer to 5 + 15", :user
+bot.chat "Tell me the answer to (5 + 15) * 2", :user
+bot.chat "Tell me the answer to ((5 + 15) * 2) / 10", :user
+bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
 
 ##
 # [system] You are my math assistant.
@@ -73,128 +67,324 @@ convo.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
 #             The answer to ((5 + 15) * 2) / 10 is 4.
 ```
 
-#### Prompts
-
-Both lazy and non-lazy conversations accept text as a prompt.
-Depending on the provider, they may also accept a
-[URI](https://docs.ruby-lang.org/en/master/URI.html)
-or
-[LLM::File](https://0x1eef.github.io/x/llm/LLM/File.html)
-object. Generally a
-[URI](https://docs.ruby-lang.org/en/master/URI.html)
-object is used to reference an image on the web, and an
-[LLM::File](https://0x1eef.github.io/x/llm/LLM/File.html)
-object is used to reference a file on the local filesystem.
-The following list shows the types of prompts that each
-provider accepts:
-
-* OpenAI       =>   String, URI
-* Gemini        =>   String, LLM::File
-* Anthropic   =>   String, URI
-* Ollama        =>   String, URI
+#### Responses
+
+The responses API is a recent addition
+[provided by OpenAI](https://platform.openai.com/docs/guides/conversation-state?api-mode=responses)
+that lets a client store message state on their servers – and in turn
+a client can avoid maintaining state manually as well as avoid sending
+the entire conversation with each request that is made. Although it is
+primarily supported by OpenAI at the moment, we might see other providers
+support it in the future. For now
+[llm.rb supports the responses API](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Responses.html)
+for the OpenAI provider:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(ENV["KEY"])
+bot = LLM::Chat.new(llm).lazy
+bot.respond File.read("./share/llm/prompts/system.txt"), :developer
+bot.respond "Tell me the answer to 5 + 15", :user
+bot.respond "Tell me the answer to (5 + 15) * 2", :user
+bot.respond "Tell me the answer to ((5 + 15) * 2) / 10", :user
+bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
+
+##
+# [developer] You are my math assistant.
+#             I will provide you with (simple) equations.
+#             You will provide answers in the format "The answer to <equation> is <answer>".
+#             I will provide you a set of messages. Reply to all of them.
+#             A message is considered unanswered if there is no corresponding assistant response.
+#
+# [user] Tell me the answer to 5 + 15
+# [user] Tell me the answer to (5 + 15) * 2
+# [user] Tell me the answer to ((5 + 15) * 2) / 10
+#
+# [assistant] The answer to 5 + 15 is 20.
+#             The answer to (5 + 15) * 2 is 40.
+#             The answer to ((5 + 15) * 2) / 10 is 4.
+```
+
+### Audio
+
+#### Speech
+
+Some but not all providers implement audio generation capabilities that
+can create speech from text, transcribe audio to text, or translate
+audio to text (usually English). The following example uses the OpenAI provider
+to create an audio file from a text prompt. The audio is then moved to
+`${HOME}/hello.mp3` as the final step. As always, consult the provider's
+documentation (eg [OpenAI docs](https://platform.openai.com/docs/api-reference/audio/create))
+for more information on how to use the audio generation API:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.audio.create_speech(input: "Hello world")
+File.binwrite File.join(Dir.home, "hello.mp3"),
+	          res.audio.string
+```
+
+#### Transcribe
+
+The following example transcribes an audio file to text. The audio file
+(`${HOME}/hello.mp3`) was theoretically created in the previous example,
+and the result is printed to the console. The example uses the OpenAI
+provider to transcribe the audio file. As always, consult the provider's
+documentation (eg
+[OpenAI docs](https://platform.openai.com/docs/api-reference/audio/createTranscription),
+[Gemini docs](https://ai.google.dev/gemini-api/docs/audio))
+for more information on how to use the audio transcription API.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Audio](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Audio.html),
+[LLM::OpenAI::Audio](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Audio.html)):
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.audio.create_transcription(
+  file: LLM::File(File.join(Dir.home, "hello.mp3"))
+)
+print res.text, "\n" # => "Hello world."
+```
+
+#### Translate
+
+The following example translates an audio file to text. In this example
+the audio file (`${HOME}/bomdia.mp3`) is theoretically in Portuguese,
+and it is translated to English. The example uses the OpenAI provider,
+and at the time of writing, it can only translate to English. As always,
+consult the provider's documentation (eg
+[OpenAI docs](https://platform.openai.com/docs/api-reference/audio/createTranslation),
+[Gemini docs](https://ai.google.dev/gemini-api/docs/audio))
+for more information on how to use the audio translation API.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Audio](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Audio.html),
+[LLM::OpenAI::Audio](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Audio.html)):
+
+
+```ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.audio.create_translation(
+  file: LLM::File(File.join(Dir.home, "bomdia.mp3"))
+)
+print res.text, "\n" # => "Good morning."
+```
+
+### Images
+
+#### Create
+
+Some but not all LLM providers implement image generation capabilities that
+can create new images from a prompt, or edit an existing image with a
+prompt. The following example uses the OpenAI provider to create an
+image of a dog on a rocket to the moon. The image is then moved to
+`${HOME}/dogonrocket.png` as the final step.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Images](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Images.html),
+[LLM::OpenAI::Images](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Images.html)):
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.images.create(prompt: "a dog on a rocket to the moon")
+res.urls.each do |url|
+  FileUtils.mv OpenURI.open_uri(url).path,
+               File.join(Dir.home, "dogonrocket.png")
+end
+```
+
+#### Edit
+
+The following example is focused on editing a local image with the aid
+of a prompt. The image (`/images/cat.png`) is returned to us with the cat
+now wearing a hat. The image is then moved to `${HOME}/catwithhat.png` as
+the final step.
+
+Results and quality may vary, consider prompt adjustments if the results
+are not satisfactory, and consult the provider's documentation
+(eg
+[OpenAI docs](https://platform.openai.com/docs/api-reference/images/createEdit),
+[Gemini docs](https://ai.google.dev/gemini-api/docs/image-generation))
+for more information on how to use the image editing API.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Images](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Images.html),
+[LLM::OpenAI::Images](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Images.html)):
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.images.edit(
+  image: LLM::File("/images/cat.png"),
+  prompt: "a cat with a hat",
+)
+res.urls.each do |url|
+  FileUtils.mv OpenURI.open_uri(url).path,
+               File.join(Dir.home, "catwithhat.png")
+end
+```
+
+#### Variations
+
+The following example is focused on creating variations of a local image.
+The image (`/images/cat.png`) is returned to us with five different variations.
+The images are then moved to `${HOME}/catvariation0.png`, `${HOME}/catvariation1.png`
+and so on as the final step. Consult the provider's documentation
+(eg [OpenAI docs](https://platform.openai.com/docs/api-reference/images/createVariation))
+for more information on how to use the image variations API:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+require "open-uri"
+require "fileutils"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.images.create_variation(
+  image: LLM::File("/images/cat.png"),
+  n: 5
+)
+res.urls.each.with_index do |url, index|
+  FileUtils.mv OpenURI.open_uri(url).path,
+               File.join(Dir.home, "catvariation#{index}.png")
+end
+```
+
+### Files
+
+#### Create
+
+Most LLM providers provide a Files API where you can upload files
+that can be referenced from a prompt and llm.rb has first-class support
+for this feature. The following example uses the OpenAI provider to describe
+the contents of a PDF file after it has been uploaded. The file (an instance
+of [LLM::Response::File](https://0x1eef.github.io/x/llm.rb/LLM/Response/File.html))
+is passed directly to the chat method, and generally any object a prompt supports
+can be given to the chat method.
+
+Please also see provider-specific documentation for more provider-specific
+examples and documentation
+(eg
+[LLM::Gemini::Files](https://0x1eef.github.io/x/llm.rb/LLM/Gemini/Files.html),
+[LLM::OpenAI::Files](https://0x1eef.github.io/x/llm.rb/LLM/OpenAI/Files.html)):
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(ENV["KEY"])
+bot = LLM::Chat.new(llm).lazy
+file = llm.files.create(file: LLM::File("/documents/openbsd_is_awesome.pdf"))
+bot.chat(file)
+bot.chat("What is this file about?")
+bot.messages.select(&:assistant?).each { print "[#{_1.role}] ", _1.content, "\n" }
+
+##
+# [assistant] This file is about OpenBSD, a free and open-source Unix-like operating system
+#             based on the Berkeley Software Distribution (BSD). It is known for its
+#             emphasis on security, code correctness, and code simplicity. The file
+#             contains information about the features, installation, and usage of OpenBSD.
+```
 
 ### Embeddings
 
 #### Text
 
 The
-[`LLM::Provider#embed`](https://0x1eef.github.io/x/llm/LLM/Provider.html#embed-instance_method)
+[`LLM::Provider#embed`](https://0x1eef.github.io/x/llm.rb/LLM/Provider.html#embed-instance_method)
 method generates a vector representation of one or more chunks
 of text. Embeddings capture the semantic meaning of text &ndash;
 a common use-case for them is to store chunks of text in a
 vector database, and then to query the database for *semantically
 similar* text. These chunks of similar text can then support the
 generation of a prompt that is used to query a large language model,
-which will go on to generate a response.
-
-For example, a user query might find similar text that adds important
-context to the prompt that informs the large language model in how to respond.
-The chunks of text may also carry metadata that can be used to further filter
-and contextualize the search results. This technique is popularly known as
-retrieval-augmented generation (RAG). Embeddings can also be used for
-other purposes as well &ndash; RAG is just one of the most popular use-cases.
-
-Let's take a look at an example that generates a couple of vectors
-for two chunks of text:
+which will go on to generate a response:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
 llm = LLM.openai(ENV["KEY"])
-res = llm.embed(["programming is fun", "ruby is a programming language"])
+res = llm.embed(["programming is fun", "ruby is a programming language", "sushi is art"])
 print res.class, "\n"
 print res.embeddings.size, "\n"
 print res.embeddings[0].size, "\n"
 
 ##
 # LLM::Response::Embedding
-# 2
+# 3
 # 1536
 ```
 
-### LLM
-
-#### Timeouts
-
-When running the ollama provider locally it might take a while for
-the language model to reply &ndash; depending on hardware and the
-size of the model. The following example demonstrates how to wait
-a longer period of time for a response through the use of the
-`timeout` configuration option with the `qwq` model. The following
-example waits up to 15 minutes for a response:
-
-```ruby
-#!/usr/bin/env ruby
-require "llm"
+### Memory
 
-llm = LLM.ollama(nil, timeout: 60*15)
-llm.chat "What is the meaning of life ?", model: "qwq"
-llm.last_message.tap { print "[assistant] ", _1.content, "\n" }
-```
+#### Child process
 
-#### Models
+When it comes to the generation of audio, images, and video memory consumption
+can be a potential problem. There are a few strategies in place to deal with this,
+and one lesser known strategy is to let a child process handle the memory cost
+by delegating media generation to a child process.
 
-Generally each Large Language Model provides multiple models to choose
-from, and each model has its own set of capabilities and limitations.
-The following example demonstrates how to query the list of models
-through the
-[LLM::Provider#models](http://0x1eef.github.io/x/llm/LLM/Provider.html#models-instance_method)
-method &ndash; the example happens to use the ollama provider but
-this can be done for any provider:
+Once a child process exits, any memory it had used is freed immediately and
+the parent process can continue to have a small memory footprint. In a sense
+it is similar to being able to use malloc + free from Ruby. The following example
+demonstrates how that might look like in practice:
 
 ```ruby
 #!/usr/bin/env ruby
 require "llm"
 
-##
-# List models
-llm = LLM.ollama(nil)
-llm.models.each { print "#{_2.name}: #{_2.description}", "\n" }
-
-##
-# Select a model
-llm.chat "Hello, world!", model: llm.models["qwq"]
-
-##
-# This also works
-llm.chat "Hello, world!", model: "qwq"
+llm = LLM.gemini(ENV["KEY"])
+fork do
+  %w[dog cat sheep goat capybara].each do |animal|
+    res = llm.images.create(prompt: "a #{animal} on a rocket to the moon")
+    File.binwrite "#{animal}.png", res.images[0].binary
+  end
+end
+Process.wait
 ```
-## Providers
 
-- [x] [Anthropic](https://www.anthropic.com/)
-- [x] [OpenAI](https://platform.openai.com/docs/overview)
-- [x] [Gemini](https://ai.google.dev/gemini-api/docs)
-- [x] [Ollama](https://github.com/ollama/ollama#readme)
-- [ ] Hugging Face
-- [ ] Cohere
-- [ ] AI21 Labs
-- [ ] Replicate
-- [ ] Mistral AI
+## API reference
 
-## Documentation
+The README tries to provide a high-level overview of the library. For everything
+else there's the API reference. It covers classes and methods that the README glances
+over or doesn't cover at all. The API reference is available at
+[0x1eef.github.io/x/llm.rb](https://0x1eef.github.io/x/llm.rb).
 
-A complete API reference is available at [0x1eef.github.io/x/llm](https://0x1eef.github.io/x/llm)
 
 ## Install
 
@@ -202,6 +392,24 @@ llm.rb can be installed via rubygems.org:
 
 	gem install llm.rb
 
+## Philosophy
+
+llm.rb was built for developers who believe that simplicity is strength.
+It provides a clean, dependency-free interface to Large Language Models,
+treating Ruby itself as the primary platform &ndash; not Rails or any other
+specific framework or library. There is no hidden magic or extreme
+metaprogramming.
+
+Every part of llm.rb is designed to be explicit, composable, memory-safe,
+and production-ready without compromise. No unnecessary abstractions,
+no global configuration, and no dependencies that aren't part of standard
+Ruby. It has been inspired in part by other languages such as Python, but
+it is not a port of any other library.
+
+Good software doesn’t need marketing. It just needs to work. :)
+
 ## License
 
-MIT. See [LICENSE.txt](LICENSE.txt) for more details
+[BSD Zero Clause](https://choosealicense.com/licenses/0bsd/)
+<br>
+See [LICENSE](./LICENSE)

data/lib/llm/buffer.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+module LLM
+  ##
+  # @private
+  # {LLM::Buffer LLM::Buffer} provides an Enumerable object that
+  # yields each message in a conversation on-demand, and only sends
+  # a request to the LLM when a response is needed.
+  class Buffer
+    include Enumerable
+
+    ##
+    # @param [LLM::Provider] provider
+    # @return [LLM::Buffer]
+    def initialize(provider)
+      @provider = provider
+      @pending = []
+      @completed = []
+    end
+
+    ##
+    # @yield [LLM::Message]
+    #  Yields each message in the conversation thread
+    # @raise (see LLM::Provider#complete)
+    # @return [void]
+    def each
+      empty! unless @pending.empty?
+      @completed.each { yield(_1) }
+    end
+
+    ##
+    # @param [[LLM::Message, Hash]] item
+    #  A message and its parameters
+    # @return [void]
+    def <<(item)
+      @pending << item
+      self
+    end
+    alias_method :push, :<<
+
+    ##
+    # @return [String]
+    def inspect
+      "#<#{self.class.name}:0x#{object_id.to_s(16)} " \
+      "completed_count=#{@completed.size} pending_count=#{@pending.size}>"
+    end
+
+    private
+
+    def empty!
+      message, params, method = @pending[-1]
+      if method == :complete
+        complete!(message, params)
+      elsif method == :respond
+        respond!(message, params)
+      else
+        raise LLM::Error, "Unknown method: #{method}"
+      end
+    end
+
+    def complete!(message, params)
+      messages = @pending[0..-2].map { _1[0] }
+      completion = @provider.complete(
+        message.content,
+        message.role,
+        **params.merge(messages:)
+      )
+      @completed.concat([*messages, message, completion.choices[0]])
+      @pending.clear
+    end
+
+    def respond!(message, params)
+      input = @pending[0..-2].map { _1[0] }
+      @response = @provider.responses.create(
+        message.content,
+        message.role,
+        **params.merge(input:).merge(@response ? {previous_response_id: @response.id} : {})
+      )
+      @completed.concat([*input, message, @response.outputs[0]])
+      @pending.clear
+    end
+  end
+end