llm.rb 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 5cd6331d31fab0e7582d9a0bef7a10b45e910fff70faace6bae774dd2757cff6
+  data.tar.gz: 42f0d68055c3f4aa732bb31b7dddd8d72b8e4fa90ba8727e871a43b736060e8f
+SHA512:
+  metadata.gz: b3158f6d82d9f344deef6727ae2e74f2d7fc8637c46188d2aaf7c92d395cd50e737f39674d283decb17bcd9b7c45f5a0b592bf2fe3d48a8f121e89cdc0b5be35
+  data.tar.gz: 74a9dbe6e7a2082cf764db07058761ed2ca5fb36916e0f78d487b4648fd34b531ca801e2202706a138d45191ed8f0f7e05db357f400aa05569403eadfb8fc887

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2024 Antar Azri
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,146 @@
+## About
+
+llm.rb is a lightweight library that provides a common interface
+and set of functionality for multiple Large Language Models (LLMs). It
+is designed to be simple, flexible, and easy to use.
+
+## Examples
+
+### Providers
+
+#### LLM::Provider
+
+All providers inherit from [LLM::Provider](https://0x1eef.github.io/x/llm/LLM/Provider.html) –
+they share a common interface and set of functionality. Each provider can be instantiated
+using an API key (if required) and an optional set of configuration options via
+[the singleton methods of LLM](https://0x1eef.github.io/x/llm/LLM.html). For example:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai("yourapikey")
+llm = LLM.gemini("yourapikey")
+llm = LLM.anthropic("yourapikey")
+llm = LLM.ollama(nil)
+```
+
+### Completions
+
+#### Conversation
+
+The
+[LLM::Provider#chat](https://0x1eef.github.io/x/llm/LLM/Provider.html#chat-instance_method)
+method returns a
+[LLM::LazyConversation](https://0x1eef.github.io/x/llm/LLM/LazyConversation.html)
+object, and it allows for a "lazy" conversation where messages are batched and
+sent to the provider only when necessary. The non-lazy counterpart is available via the
+[LLM::Provider#chat!](https://0x1eef.github.io/x/llm/LLM/Provider.html#chat!-instance_method)
+method.
+
+Both lazy and non-lazy conversations maintain a message thread that can
+be reused as context throughout a conversation. For the sake of brevity the system
+prompt is loaded from
+[a file](./share/llm/prompts/system.txt)
+in the following example – all other prompts are "user" prompts –
+and a single request is made to the provider when iterating over the messages
+belonging to a lazy conversation:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(ENV["KEY"])
+bot = llm.chat File.read("./share/llm/prompts/system.txt"), :system
+bot.chat "What color is the sky?"
+bot.chat "What color is an orange?"
+bot.chat "I like Ruby"
+bot.messages.each { print "[#{_1.role}] ", _1.content, "\n" }
+
+##
+# [system] You are a friendly chatbot. Sometimes, you like to tell a joke.
+#          But the joke must be based on the given inputs.
+#          I will provide you a set of messages. Reply to all of them.
+#          A message is considered unanswered if there is no corresponding assistant response.
+#
+# [user] What color is the sky?
+# [user] What color is an orange?
+# [user] I like Ruby
+#
+# [assistant] The sky is typically blue during the day. As for an orange,
+#             it is usually orange in color—funny how that works, right?
+#             I love Ruby too! Speaking of colors, why did the orange stop?
+#             Because it ran out of juice! 🍊😂
+```
+
+#### Prompts
+
+Both lazy and non-lazy conversations accept text as a prompt.
+Depending on the provider, they may also accept a
+[URI](https://docs.ruby-lang.org/en/master/URI.html)
+or
+[LLM::File](https://0x1eef.github.io/x/llm/LLM/File.html)
+object. Generally a
+[URI](https://docs.ruby-lang.org/en/master/URI.html)
+object is used to reference an image on the web, and an
+[LLM::File](https://0x1eef.github.io/x/llm/LLM/File.html)
+object is used to reference a file on the local filesystem.
+The following list shows the types of prompts that each
+provider accepts:
+
+* OpenAI       =>   String, URI
+* Gemini        =>   String, LLM::File
+* Anthropic   =>   String, URI
+* Ollama        =>   String, URI
+
+### Embeddings
+
+#### Text
+
+The
+[`LLM::Provider#embed`](https://0x1eef.github.io/x/llm/LLM/Provider.html#embed-instance_method)
+method generates a vector representation of a given piece of text.
+Embeddings capture the semantic meaning of text, and they are
+commonly used in tasks such as text similarity comparison (e.g., finding related documents),
+semantic search in vector databases, and the clustering and classification
+of text-based data:
+
+```ruby
+#!/usr/bin/env ruby
+require "llm"
+
+llm = LLM.openai(ENV["KEY"])
+res = llm.embed("Hello, world!")
+print res.class, "\n"
+print res.embeddings.size, "\n"
+print res.embeddings[0].size, "\n"
+
+##
+# LLM::Response::Embedding
+# 1
+# 1536
+```
+
+## Providers
+
+- [x] [Anthropic](https://www.anthropic.com/)
+- [x] [OpenAI](https://platform.openai.com/docs/overview)
+- [x] [Gemini](https://ai.google.dev/gemini-api/docs)
+- [x] [Ollama](https://github.com/ollama/ollama#readme)
+- [ ] Hugging Face
+- [ ] Cohere
+- [ ] AI21 Labs
+- [ ] Replicate
+- [ ] Mistral AI
+
+## Documentation
+
+A complete API reference is available at [0x1eef.github.io/x/llm](https://0x1eef.github.io/x/llm)
+
+## Install
+
+LLM has not been published to RubyGems.org yet. Stay tuned
+
+## License
+
+MIT. See [LICENSE.txt](LICENSE.txt) for more details

data/lib/llm/conversation.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+module LLM
+  ##
+  # {LLM::Conversation LLM::Conversation} provides a conversation
+  # object that maintains a thread of messages that act as the
+  # context of the conversation.
+  #
+  # @example
+  #   llm = LLM.openai(key)
+  #   bot = llm.chat("What is the capital of France?")
+  #   bot.chat("What should we eat in Paris?")
+  #   bot.chat("What is the weather like in Paris?")
+  #   p bot.messages.map { [_1.role, _1.content] }
+  class Conversation
+    ##
+    # @return [Array<LLM::Message>]
+    attr_reader :messages
+
+    ##
+    # @param [LLM::Provider] provider
+    #  A provider
+    def initialize(provider)
+      @provider = provider
+      @messages = []
+    end
+
+    ##
+    # @param prompt (see LLM::Provider#prompt)
+    # @return [LLM::Conversation]
+    def chat(prompt, role = :user, **params)
+      tap do
+        completion = @provider.complete(prompt, role, **params.merge(messages:))
+        @messages.concat [Message.new(role, prompt), completion.choices[0]]
+      end
+    end
+  end
+end

data/lib/llm/core_ext/ostruct.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+require "ostruct"
+class OpenStruct
+  module FromHash
+    ##
+    # @example
+    #   obj = OpenStruct.from_hash(person: {name: 'John'})
+    #   obj.person.name  # => 'John'
+    #   obj.person.class # => OpenStruct
+    # @param [Hash] hash_obj
+    #   A Hash object
+    # @return [OpenStruct]
+    #   An OpenStruct object initialized by visiting `hash_obj` with
+    #   recursion
+    def from_hash(hash_obj)
+      visited_object = {}
+      hash_obj.each do |key, value|
+        visited_object[key] = walk(value)
+      end
+      OpenStruct.new(visited_object)
+    end
+
+    private
+
+    def walk(value)
+      if Hash === value
+        from_hash(value)
+      elsif Array === value
+        value.map { |v| (Hash === v) ? from_hash(v) : v }
+      else
+        value
+      end
+    end
+  end
+  extend FromHash
+end

data/lib/llm/error.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+module LLM
+  ##
+  # The superclass of all LLM errors
+  class Error < RuntimeError
+    def initialize
+      block_given? ? yield(self) : nil
+    end
+
+    ##
+    # The superclass of all HTTP protocol errors
+    class BadResponse < Error
+      ##
+      # @return [Net::HTTPResponse]
+      #  Returns the response associated with an error
+      attr_accessor :response
+    end
+
+    ##
+    # HTTPUnauthorized
+    Unauthorized = Class.new(BadResponse)
+
+    ##
+    # HTTPTooManyRequests
+    RateLimit = Class.new(BadResponse)
+  end
+end

data/lib/llm/file.rb

@@ -0,0 +1,66 @@
+# frozen_string_literal: true
+
+class LLM::File
+  ##
+  # @return [Hash]
+  #  Returns a hash of common file extensions and their
+  #  corresponding MIME types
+  def self.mime_types
+    @mime_types ||= {
+      # Images
+      ".png" => "image/png",
+      ".jpg" => "image/jpeg",
+      ".jpeg" => "image/jpeg",
+      ".webp" => "image/webp",
+
+      # Videos
+      ".flv" => "video/x-flv",
+      ".mov" => "video/quicktime",
+      ".mpeg" => "video/mpeg",
+      ".mpg" => "video/mpeg",
+      ".mp4" => "video/mp4",
+      ".webm" => "video/webm",
+      ".wmv" => "video/x-ms-wmv",
+      ".3gp" => "video/3gpp",
+
+      # Audio
+      ".aac" => "audio/aac",
+      ".flac" => "audio/flac",
+      ".mp3" => "audio/mpeg",
+      ".m4a" => "audio/mp4",
+      ".mpga" => "audio/mpeg",
+      ".opus" => "audio/opus",
+      ".pcm" => "audio/L16",
+      ".wav" => "audio/wav",
+      ".weba" => "audio/webm",
+
+      # Documents
+      ".pdf" => "application/pdf",
+      ".txt" => "text/plain"
+    }.freeze
+  end
+
+  ##
+  # @return [String]
+  #  Returns the path to a file
+  attr_reader :path
+
+  def initialize(path)
+    @path = path
+  end
+
+  ##
+  # @return [String]
+  #  Returns the MIME type of the file
+  def mime_type
+    self.class.mime_types[File.extname(path)]
+  end
+end
+
+##
+# @param [String] path
+#  The path to a file
+# @return [LLM::File]
+def LLM.File(path)
+  LLM::File.new(path)
+end

data/lib/llm/http_client.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module LLM
+  module HTTPClient
+    require "net/http"
+    ##
+    # Initiates a HTTP request
+    # @param [Net::HTTP] http
+    #  The HTTP object to use for the request
+    # @param [Net::HTTPRequest] req
+    #  The request to send
+    # @return [Net::HTTPResponse]
+    #  The response from the server
+    # @raise [LLM::Error::Unauthorized]
+    #  When authentication fails
+    # @raise [LLM::Error::RateLimit]
+    #  When the rate limit is exceeded
+    # @raise [LLM::Error::BadResponse]
+    #  When any other unsuccessful status code is returned
+    # @raise [SystemCallError]
+    #  When there is a network error at the operating system level
+    def request(http, req)
+      res = http.request(req)
+      res.tap(&:value)
+    rescue Net::HTTPClientException
+      error_handler.new(res).raise_error!
+    end
+  end
+end

data/lib/llm/lazy_conversation.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+module LLM
+  require_relative "message_queue"
+
+  ##
+  # {LLM::LazyConversation LLM::LazyConversation} provides a
+  # conversation object that allows input prompts to be queued
+  # and only sent to the LLM when a response is needed.
+  #
+  # @example
+  #   llm = LLM.openai(key)
+  #   bot = llm.chat("Be a helpful weather assistant", :system)
+  #   bot.chat("What's the weather like in Rio?")
+  #   bot.chat("What's the weather like in Algiers?")
+  #   bot.messages.each do |message|
+  #     # A single request is made at this point
+  #   end
+  class LazyConversation
+    ##
+    # @return [LLM::MessageQueue]
+    attr_reader :messages
+
+    ##
+    # @param [LLM::Provider] provider
+    #  A provider
+    def initialize(provider)
+      @provider = provider
+      @messages = LLM::MessageQueue.new(provider)
+    end
+
+    ##
+    # @param prompt (see LLM::Provider#prompt)
+    # @return [LLM::Conversation]
+    def chat(prompt, role = :user, **params)
+      tap { @messages << [prompt, role, params] }
+    end
+  end
+end

data/lib/llm/message.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module LLM
+  class Message
+    ##
+    # @return [Symbol]
+    attr_reader :role
+
+    ##
+    # @return [String]
+    attr_reader :content
+
+    ##
+    # @return [Hash]
+    attr_reader :extra
+
+    ##
+    # @param [Symbol] role
+    # @param [String] content
+    # @param [Hash] extra
+    # @return [LLM::Message]
+    def initialize(role, content, extra = {})
+      @role = role
+      @content = content
+      @extra = extra
+    end
+
+    ##
+    # @return [OpenStruct]
+    def logprobs
+      return nil unless extra.key?(:logprobs)
+      OpenStruct.from_hash(extra[:logprobs])
+    end
+
+    ##
+    # @return [Hash]
+    def to_h
+      {role:, content:}
+    end
+
+    ##
+    # @param [Object] other
+    #  The other object to compare
+    # @return [Boolean]
+    #  Returns true when the "other" object has the same role and content
+    def ==(other)
+      if other.respond_to?(:to_h)
+        to_h == other.to_h
+      else
+        false
+      end
+    end
+    alias_method :eql?, :==
+  end
+end

data/lib/llm/message_queue.rb

@@ -0,0 +1,47 @@
+# frozen_string_literal: true
+
+module LLM
+  ##
+  # {LLM::MessageQueue LLM::MessageQueue} provides an Enumerable
+  # object that yields each message in a conversation on-demand,
+  # and only sends a request to the LLM when a response is needed.
+  class MessageQueue
+    include Enumerable
+
+    ##
+    # @param [LLM::Provider] provider
+    # @return [LLM::MessageQueue]
+    def initialize(provider)
+      @provider = provider
+      @messages = []
+    end
+
+    ##
+    # @yield [LLM::Message]
+    #  Yields each message in the conversation thread
+    # @raise (see LLM::Provider#complete)
+    # @return [void]
+    def each
+      @messages = complete! unless @messages.grep(LLM::Message).size == @messages.size
+      @messages.each { yield(_1) }
+    end
+
+    ##
+    # @param message [Object]
+    #  A message to add to the conversation thread
+    # @return [void]
+    def <<(message)
+      @messages << message
+    end
+    alias_method :push, :<<
+
+    private
+
+    def complete!
+      prompt, role, params = @messages[-1]
+      rest = @messages[0..-2].map { (Array === _1) ? LLM::Message.new(_1[1], _1[0]) : _1 }
+      comp = @provider.complete(prompt, role, **params.merge(messages: rest)).choices.last
+      [*rest, LLM::Message.new(role, prompt), comp]
+    end
+  end
+end

data/lib/llm/provider.rb

@@ -0,0 +1,114 @@
+# frozen_string_literal: true
+
+module LLM
+  require "llm/http_client"
+  ##
+  # The Provider class represents an abstract class for
+  # LLM (Language Model) providers
+  class Provider
+    include HTTPClient
+    ##
+    # @param [String] secret
+    #  The secret key for authentication
+    # @param [String] host
+    #  The host address of the LLM provider
+    # @param [Integer] port
+    #  The port number
+    def initialize(secret, host:, port: 443, ssl: true)
+      @secret = secret
+      @http = Net::HTTP.new(host, port).tap do |http|
+        http.use_ssl = ssl
+      end
+    end
+
+    ##
+    # Returns an inspection of the provider object
+    # @return [String]
+    # @note The secret key is redacted in inspect for security reasons
+    def inspect
+      "#<#{self.class.name}:0x#{object_id.to_s(16)} @secret=[REDACTED] @http=#{@http.inspect}>"
+    end
+
+    ##
+    # @param [String] input
+    #  The input to embed
+    # @raise [NotImplementedError]
+    #  When the method is not implemented by a subclass
+    # @return [LLM::Response::Embedding]
+    def embed(input, **params)
+      raise NotImplementedError
+    end
+
+    ##
+    # Completes a given prompt using the LLM
+    # @param [String] prompt
+    #  The input prompt to be completed
+    # @param [Symbol] role
+    #  The role of the prompt (e.g. :user, :system)
+    # @raise [NotImplementedError]
+    #  When the method is not implemented by a subclass
+    # @return [LLM::Response::Completion]
+    def complete(prompt, role = :user, **params)
+      raise NotImplementedError
+    end
+
+    ##
+    # Starts a new lazy conversation
+    # @param prompt (see LLM::Provider#complete)
+    # @param role (see LLM::Provider#complete)
+    # @raise (see LLM::Provider#complete)
+    # @return [LLM::LazyConversation]
+    def chat(prompt, role = :user, **params)
+      LazyConversation.new(self).chat(prompt, role, **params)
+    end
+
+    ##
+    # Starts a new conversation
+    # @param prompt (see LLM::Provider#complete)
+    # @param role (see LLM::Provider#complete)
+    # @raise (see LLM::Provider#complete)
+    # @return [LLM::Conversation]
+    def chat!(prompt, role = :user, **params)
+      Conversation.new(self).chat(prompt, role, **params)
+    end
+
+    private
+
+    ##
+    # Prepares a request for authentication
+    # @param [Net::HTTP::Request] req
+    #  The request to prepare for authentication
+    # @raise [NotImplementedError]
+    #  (see LLM::Provider#complete)
+    def auth(req)
+      raise NotImplementedError
+    end
+
+    ##
+    # @return [Module]
+    #  Returns the module responsible for parsing a successful LLM response
+    # @raise [NotImplementedError]
+    #  (see LLM::Provider#complete)
+    def response_parser
+      raise NotImplementedError
+    end
+
+    ##
+    # @return [Class]
+    #  Returns the class responsible for handling an unsuccessful LLM response
+    # @raise [NotImplementedError]
+    #  (see LLM::Provider#complete)
+    def error_handler
+      raise NotImplementedError
+    end
+
+    ##
+    # Prepares a request before sending it
+    def preflight(req, body)
+      req.content_type = "application/json"
+      req.body = JSON.generate(body)
+      auth(req)
+      req
+    end
+  end
+end

data/lib/llm/providers/anthropic/error_handler.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+class LLM::Anthropic
+  class ErrorHandler
+    ##
+    # @return [Net::HTTPResponse]
+    #  Non-2XX response from the server
+    attr_reader :res
+
+    ##
+    # @param [Net::HTTPResponse] res
+    #  The response from the server
+    # @return [LLM::Anthropic::ErrorHandler]
+    def initialize(res)
+      @res = res
+    end
+
+    ##
+    # @raise [LLM::Error]
+    #  Raises a subclass of {LLM::Error LLM::Error}
+    def raise_error!
+      case res
+      when Net::HTTPForbidden
+        raise LLM::Error::Unauthorized.new { _1.response = res }, "Authentication error"
+      when Net::HTTPTooManyRequests
+        raise LLM::Error::RateLimit.new { _1.response = res }, "Too many requests"
+      else
+        raise LLM::Error::BadResponse.new { _1.response = res }, "Unexpected response"
+      end
+    end
+  end
+end

data/lib/llm/providers/anthropic/format.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+class LLM::Anthropic
+  module Format
+    ##
+    # @param [Array<LLM::Message>] messages
+    #  The messages to format
+    # @return [Array<Hash>]
+    def format(messages)
+      messages.map { {role: _1.role, content: format_content(_1.content)} }
+    end
+
+    private
+
+    ##
+    # @param [String, URI] content
+    #  The content to format
+    # @return [String, Hash]
+    #  The formatted content
+    def format_content(content)
+      if URI === content
+        [{
+          type: :image,
+          source: {type: :base64, media_type: LLM::File(content.to_s).mime_type, data: [content.to_s].pack("m0")}
+        }]
+      else
+        content
+      end
+    end
+  end
+end