ai_client 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2453f748447f37e755f0087615c845570fe3ea3c4bd06a687947dceedee3e89b
-  data.tar.gz: 79359bcd209448add248514d9a8ee5dc4e0fa69833666df0e779715b574d450a
+  metadata.gz: 72666a60d1c49346ce8512fbfe97fbc163801f5cc22e9f7ed3883c3c8e28901b
+  data.tar.gz: bbcaee95e9ef2a6fd8bad5159ff8aa6d8cc88f443302dd37573c0bf7da512a66
 SHA512:
-  metadata.gz: 28007804ea1e223b22846cc199c4bd14d7349f5e051bbc7007ced3641e7174c3fac59b7a225bc4926ffc68d5018a35ed96a8809bf959c8007e978bc831770b9a
-  data.tar.gz: 18ce04f9f83d0c12caadab051b81c48f7dbafa73ff0b2a34df28e3ddfb5ff3c088dcec019a15bc2f2dfc909bb6db0a308b512b6711baaab058629705a1448040
+  metadata.gz: 04045a6f8a78671930429f9481e4319b36cf5955d802f8f6ae5b4ec6c4bd1b4c88d4854bcaa44fbb3b4a0f4b7d68c7a4f3d0c4045c767c590cf417efac905e5f
+  data.tar.gz: d26f1770c0dd38d6c83fed91eca4e8489ac485415b91bb0e0d5ef0496f09af20f0e532974b984fcabedd997562494f20881966b3b5bee0e1a091013431f5e18d

data/CHANGELOG.md

@@ -2,6 +2,21 @@
 
 ## Released
 
+### [0.4.0] - 2024-10-20
+- Removed Logger.new(STDOUT) from the default configuration
+  > config.logger now returns nil  If you want a class or instance logger setup, then you will have to do a config.logger = Logger.new(STDOUT) or whatever you need.
+- Adding basic @context for chat-bots
+- Added `context_length` to configuration as the number of responses to remember as context
+- Added a default model for each major provider; using "auto" for open_router.ai
+- Added a default provider (OpenAI)
+  > AiClient.new() will use the config.default_provider and that provider's default_model
+- Fixed problem with advanced block-based prompt construction for chat
+
+### [0.3.1] - 2024-10-19
+- updated the open_routed_extensions file
+- added simplecov to see code coverage
+- updated README with options doc
+
 ### [0.3.0] - 2024-10-13
 - Breaking Change
 - Added new class AiClient::Function to encapsulate the callback functions used as tools in chats.

data/README.md

@@ -2,10 +2,14 @@
 
 First and foremost a big **THANK YOU** to [Kevin Sylvestre](https://ksylvest.com/) for his gem [OmniAI](https://github.com/ksylvest/omniai) and [Olympia](https://olympia.chat/) for their [open_router gem](https://github.com/OlympiaAI/open_router) upon which this effort depends.
 
-Version 0.3.0 has a breaking change w/r/t how [Callback Functions (aka Tools)](#callback-functions-aka-tools) are defined and used.
+Version 0.4.0 has two changes which may break your existing application.
+1. The default configuration no longer has a Logger instance.  You will need to add your own instance to either the class or instance configuration using `AiClient.class_config.logger = YourLogger` and/or `client.config.logger = YourLogger`
+2. The `chat` method now keeps a context window.  The window length is defined by the configuration item `context_length`  If you do not want to maintain a context window, set the `context_length` configuration item to either nil or zero.
 
 See the  [change log](CHANGELOG.md) for recent modifications.
 
+You should also checkout the [raix gem](https://github.com/OlympiaAI/raix).  I like the way that Obie's API is setup for callback functions.  [raix-rails](https://github.com/OlympiaAI/raix-rails) is also available.
+
 
 <!-- Tocer[start]: Auto-generated, don't remove. -->
 
@@ -30,13 +34,28 @@ See the  [change log](CHANGELOG.md) for recent modifications.
         - [3. Load Complete Configuration from a YAML File](#3-load-complete-configuration-from-a-yaml-file)
     - [Top-level Client Methods](#top-level-client-methods)
         - [chat](#chat)
+            - [Cpmtext](#cpmtext)
         - [embed](#embed)
         - [speak](#speak)
         - [transcribe](#transcribe)
     - [Options](#options)
+        - [Common Options for All Methods](#common-options-for-all-methods)
+        - [Chat-specific Options](#chat-specific-options)
+        - [Embed-specific Options](#embed-specific-options)
+        - [Speak-specific Options](#speak-specific-options)
+        - [Transcribe-specific Options](#transcribe-specific-options)
     - [Advanced Prompts](#advanced-prompts)
     - [Callback Functions (aka Tools)](#callback-functions-aka-tools)
         - [Defining a Callback Function](#defining-a-callback-function)
+    - [OpenRouter Extensions and AiClient::LLM](#openrouter-extensions-and-aiclientllm)
+        - [Instance Methods](#instance-methods)
+        - [Class Methods](#class-methods)
+    - [AiClient::LLM Data Table](#aiclientllm-data-table)
+        - [Key Features](#key-features)
+        - [Class Methods](#class-methods-1)
+        - [Instance Methods](#instance-methods-1)
+        - [Usage Example](#usage-example)
+        - [Integration with ActiveHash](#integration-with-activehash)
   - [Best ?? Practices](#best--practices)
   - [OmniAI and OpenRouter](#omniai-and-openrouter)
   - [Contributing](#contributing)
@@ -96,7 +115,7 @@ AiClient.class_config.envar_api_key_bames = {
   google:       'your_envar_name',
   mistral:      'your_envar_name',
   open_router:  'your_envar_name',
-  opena:        'your_envar_name'
+  openai:       'your_envar_name'
 }
 
 AiClient.class_config.save('path/to/file.yml')
@@ -123,13 +142,49 @@ To explicitly designate a provider to use with an AiClient instance use the para
 Basic usage:
 
 ```ruby
-AI = AiClient.new('gpt-4o')
+require 'ai_client'
+ai = AiClient.new # use default model and provider
+ai.model    #=> 'gpt-4o' is the default
+ai.provider #=> :openai  is the default
+#
+# To change the class defaults:
+#
+AiClient.default_provider       = :anthropic
+AiClient.default_model[:openai] = 'gpt-4o-mini'
+#
+# To get an Array of models and providers
+#
+AiClient.models     # from open_router.ai
+AiClient.providers  # from open_router.ai
+#
+# To get details about a specific provider/model pair:
+#
+AiClient.model_details('openai/gpt-4o-mini')  # from open_router.ai
+```
+
+You can specify which model you want to use and `AiClient` will use the provider associated with that model.
+
+
+```ruby
+AI = AiClient.new('gpt-4o-mini') # sets provider to :openai
+#
+# If you want to use the open_router.ai service instead of
+# going directly to OpenAI do it this way:
+#
+AI = AiClient.new('openai/gpt-4o-mini') # sets provider to :open_router
+```
+
+Of course you could specify both the model and the provider that you want to use:
+
+
+```ruby
+AI = AiClient.new('mistral', provider: :ollama)
 ```
 
-That's it.  Just provide the model name that you want to use.  If you application is using more than one model, no worries, just create multiple AiClient instances.
+That's it.  What could be simpler?  If your application is using more than one model, no worries, just create multiple `AiClient` instances.
 
 ```ruby
-c1 = AiClient.new('nomic-embeddings-text')
+c1 = AiClient.new('nomic-embed-text')
 c2 = AiClient.new('gpt-4o-mini')
 ```
 
@@ -148,6 +203,53 @@ There are three levels of configuration, each inherenting from the level above.
 
 The file [lib/ai_client/configuration.rb] hard codes the default configuration.  This is used to update the [lib/ai_client/config.yml] file during development.  If you have some changes for this configuration please send me a pull request so we can all benefit from your efforts.
 
+```ruby
+{
+                 :logger => nil,
+                :timeout => nil,
+             :return_raw => false,
+         :context_length => 5,
+              :providers => {},
+    :envar_api_key_names => {
+          :anthropic => [
+            "ANTHROPIC_API_KEY"
+        ],
+             :google => [
+            "GOOGLE_API_KEY"
+        ],
+            :mistral => [
+            "MISTRAL_API_KEY"
+        ],
+        :open_router => [
+            "OPEN_ROUTER_API_KEY",
+            "OPENROUTER_API_KEY"
+        ],
+             :openai => [
+            "OPENAI_API_KEY"
+        ]
+    },
+      :provider_patterns => {
+          :anthropic => /^claude/i,
+             :openai => /^(gpt|chatgpt|o1|davinci|curie|babbage|ada|whisper|tts|dall-e)/i,
+             :google => /^(gemini|gemma|palm)/i,
+            :mistral => /^(mistral|codestral|mixtral)/i,
+            :localai => /^local-/i,
+             :ollama => /(llama|nomic)/i,
+        :open_router => /\//
+    },
+       :default_provider => :openai,
+          :default_model => {
+          :anthropic => "claude-3-5-sonnet-20240620",
+             :openai => "gpt-4o",
+             :google => "gemini-pro-1.5",
+            :mistral => "mistral-large",
+            :localai => "llama3.2",
+             :ollama => "llama3.2",
+        :open_router => "auto"
+    }
+}
+```
+
 #### Class Configuration
 
 The class configuration is derived initially from the default configuration.  It can be changed in three ways.
@@ -227,6 +329,16 @@ The response will be a simple string or a response object based upon the setting
 
 See the [Advanced Prompts] section to learn how to configure a complex prompt message.
 
+####### Cpmtext
+
+**context_length**
+
+The `context_length` configuration item is used to keep the last "context_length" responses within the chat context window.  If you do not want to keep a context window, you should set the value of `config.context_length = 0`  When you do either at the class or instance level, the chat response will be provided without the LLM knowing any prior context.  If you are implementing a chat-bot, you will want it to have a context of the current conversation.
+
+```ruby
+AiClient.config.context_length #=> 5
+AiClient.config.context_length = 0  # Turns off the context window
+```
 
 ##### embed
 
@@ -242,7 +354,7 @@ Recommendation: Use PostgreSQL, pg_vector and the neighbor gem.
 ##### speak
 
 ```ruby
-res[pmse = AI.speak("Isn't it nice to have a computer that will talk to you?")
+response = AI.speak("Isn't it nice to have a computer that will talk to you?")
 ```
 
 The response will contain audio data that can be played, manipulated or saved to a file.
@@ -254,17 +366,50 @@ response = AI.transcribe(...)
 ```
 
 
-
 ### Options
 
-TODO: document the options like `provider: :ollama`
+The four major methods (chat, embed, speak, and transcribe) support various options that can be passed to the underlying client code. Here's a breakdown of the common options for each method:
+
+##### Common Options for All Methods
+
+- `provider:` - Specifies the AI provider to use (e.g., `:openai`, `:anthropic`, `:google`, `:mistral`, `:ollama`, `:localai`).
+- `model:` - Specifies the model to use within the chosen provider.
+- `api_key:` - Allows passing a specific API key, overriding the default environment variable.
+- `temperature:` - Controls the randomness of the output (typically a float between 0 and 1).
+- `max_tokens:` - Limits the length of the generated response.
+
+##### Chat-specific Options
+
+- `messages:` - An array of message objects for multi-turn conversations.
+- `functions:` - An array of available functions/tools for the model to use.
+- `function_call:` - Specifies how the model should use functions ("auto", "none", or a specific function name).
+- `stream:` - Boolean to enable streaming responses.
+
+##### Embed-specific Options
+
+- `input:` - The text or array of texts to embed.
+- `dimensions:` - The desired dimensionality of the resulting embeddings (if supported by the model).
+
+##### Speak-specific Options
+
+- `voice:` - Specifies the voice to use for text-to-speech (provider-dependent).
+- `speed:` - Adjusts the speaking rate (typically a float, where 1.0 is normal speed).
+- `format:` - Specifies the audio format of the output (e.g., "mp3", "wav").
+
+##### Transcribe-specific Options
+
+- `file:` - The audio file to transcribe (can be a file path or audio data).
+- `language:` - Specifies the language of the audio (if known).
+- `prompt:` - Provides context or specific words to aid in transcription accuracy.
+
+Note: The availability and exact names of these options may vary depending on the specific provider and model being used. Always refer to the documentation of the chosen provider for the most up-to-date and accurate information on supported options.
 
 ### Advanced Prompts
 
 In more complex application providing a simple string as your prompt is not sufficient.  AiClient can take advantage of OmniAI's complex message builder.
 
 ```ruby
-client = AiClient.new 'some_model_bane'
+client = AiClient.new 'some_model_name'
 
 completion = client.chat do |prompt|
   prompt.system('You are an expert biologist with an expertise in animals.')
@@ -331,6 +476,130 @@ In this example:
 
 See the [examples/tools.rb file](examples/tools.rb) for additional examples.
 
+### OpenRouter Extensions and AiClient::LLM
+
+The `open_router.ai` API provides a service that allows you to download a JSON file containing detailed information about all of the providers and their available models. `AiClient` has saved an old copy of this information in the [`models.yml`](lib/ai_client/models.yml) file. If you want to update this file with the latest information from `open_router.ai`, you must have a valid API key.
+
+You can still use the included `models.yml` file with the `AiClient::LLM` class. The following sections describe the convenient instance and class methods that are available. See the section on `AiClient::LLM` for complete details.
+
+##### Instance Methods
+
+- **`model_details`**: Retrieves details for the current model. Returns a hash containing the model's attributes or `nil` if not found.
+
+```ruby
+client = AiClient.new('gpt-3.5-turbo')
+details = client.model_details
+details #=>
+{
+                    :id => "openai/gpt-3.5-turbo",
+                  :name => "OpenAI: GPT-3.5 Turbo",
+               :created => 1685232000,
+           :description => "GPT-3.5 Turbo is OpenAI's fastest model. It can understand and generate natural language or code, and is optimized for chat and traditional completion tasks.\n\nTraining data up to Sep 2021.",
+        :context_length => 16385,
+          :architecture => {
+             "modality" => "text->text",
+            "tokenizer" => "GPT",
+        "instruct_type" => nil
+    },
+               :pricing => {
+            "prompt" => "0.0000005",
+        "completion" => "0.0000015",
+             "image" => "0",
+           "request" => "0"
+    },
+          :top_provider => {
+               "context_length" => 16385,
+        "max_completion_tokens" => 4096,
+                 "is_moderated" => true
+    },
+    :per_request_limits => {
+            "prompt_tokens" => "40395633",
+        "completion_tokens" => "13465211"
+    }
+}
+```
+
+- **`models`**: Retrieves model names for the current provider. Returns an array of strings with the names of the models.
+
+```ruby
+models = client.models
+```
+
+##### Class Methods
+
+- **`providers`**: Retrieves all available providers. Returns an array of unique symbols representing provider names.
+
+```ruby
+available_providers = AiClient.providers
+```
+
+- **`models(substring = nil)`**: Retrieves model IDs, optionally filtered by a substring.
+
+```ruby
+available_models = AiClient.models('turbo')
+```
+
+- **`model_details(model_id)`**: Retrieves details for a specific model using its ID. Accepts a string representing the model ID. Returns a hash containing the model's attributes or `nil` if not found.
+
+```ruby
+model_info = AiClient.model_details('openai/gpt-3.5-turbo')
+```
+
+- **`reset_llm_data`**: Resets the LLM data with the available ORC models. Returns `void`.
+
+```ruby
+AiClient.reset_llm_data
+```
+
+### AiClient::LLM Data Table
+
+The `AiClient::LLM` class serves as a central point for managing information about large language models (LLMs) available via the `open_router.ai` API service. The YAML file (`models.yml`) contains information about various LLMs and their providers. To update this information to the latest available, you must have an access API key for the `open_router.ai` service.
+
+`AiClient::LLM` is a subclass of `ActiveHash::Base`, enabling it to act like and interact with `ActiveRecord::Base` defined models. Each entry in this data store is uniquely identified by an `id` in the pattern "provider/model" all lowercase without spaces.
+
+##### Key Features
+
+- **Model and Provider Extraction**:
+  - The class provides methods to extract the model name and provider from the LLM's ID.
+  - The `model` method returns the model ID derived from the ID.
+  - The `provider` method extracts the provider name as a Symbol.
+
+##### Class Methods
+
+- **`reset_llm_data`**:
+  - A class-level method that fetches the latest model data from the open_router.ai service and updates the `models.yml` file accordingly.
+
+##### Instance Methods
+
+- **`model`**:
+  - Returns the name of the model derived from the LLM's ID.
+
+```ruby
+llm_instance = AiClient::LLM.find('openai/gpt-3.5-turbo')
+puts llm_instance.model # Output: gpt-3.5-turbo
+```
+
+- **`provider`**:
+  - Returns the name of the provider associated with the LLM's ID.
+
+```ruby
+llm_instance = AiClient::LLM.find('openai/gpt-3.5-turbo')
+puts llm_instance.provider # Output: :openai
+```
+
+##### Usage Example
+
+The `AiClient::LLM` class is predominantly used to interact with different providers of LLMs. By utilizing the `model` and `provider` methods, users can seamlessly retrieve and utilize models in their applications.
+
+```ruby
+llm_instance = AiClient::LLM.find('google/bard')
+puts "Model: #{llm_instance.model}, Provider: #{llm_instance.provider}"
+```
+
+##### Integration with ActiveHash
+
+The `AiClient::LLM` class inherits from `ActiveHash::Base`, which provides an easy way to define a set of data and allows for lookups and easy manipulation of the data structure. The use of ActiveHash makes it easier to manage the LLM data effectively without needing a full database.
+
 
 ## Best ?? Practices
 

data/lib/ai_client/chat.rb

@@ -9,8 +9,17 @@ class AiClient
   #   stream:       @stream   [Proc, nil] optional
   #   tools:        @tools    [Array<OmniAI::Tool>] optional
   #   temperature:  @temperature  [Float, nil] optional
-
-  def chat(messages, **params)    
+  #
+  # Initiates a chat session.
+  #
+  # @param messages [Array<String>] the messages to send.
+  # @param params [Hash] optional parameters.
+  # @option params [Array<OmniAI::Tool>] :tools an array of tools to use.
+  # @return [String] the result from the chat.
+  #
+  # @raise [RuntimeError] if tools parameter is invalid.
+  #
+  def chat(messages='', **params, &block)    
     if params.has_key? :tools
       tools = params[:tools]
       if tools.is_a? Array
@@ -23,14 +32,62 @@ class AiClient
       params[:tools] = tools
     end
 
-    result = call_with_middlewares(:chat_without_middlewares, messages, **params)
-    @last_response = result
-    raw? ? result : content
+    @last_messages  = messages
+    messages        = add_context(messages)
+    result          = call_with_middlewares(
+                        :chat_without_middlewares, 
+                        messages, 
+                        **params,
+                        &block
+                      )
+    @last_response  = result
+    result          = raw? ? result : content
+
+    @context.push(@last_response)
+
+    result
   end
 
 
-  def chat_without_middlewares(messages, **params)
-    @client.chat(messages, model: @model, **params)
+  # Adds context to the current prompt.
+  #
+  # @param prompt [String, Array<String>] the current prompt.
+  # @return [String, Array<String>] the prompt with context added.
+  #
+  def add_context(prompt)
+    return(prompt)  if  @config.context_length.nil? || 
+                        0 == @config.context_length ||
+                        prompt.is_a?(Array)         || 
+                        @context.empty?
+
+
+    prompt << "\nUse the following context in crafting your response.\n"
+
+    @context[..config.context_length].each do |result|
+      prompt << "You previously responded with:\n"
+      prompt << "#{raw? ? result.inspect : content(result)}"
+    end
+
+    prompt
+  end
+
+
+  # Clears the current context.
+  #
+  # @return [void]
+  #
+  def clear_context
+    @context = []
   end
 
+
+  # Chats with the client without middleware processing.
+  #
+  # @param messages [Array<String>] the messages to send.
+  # @param params [Hash] optional parameters.
+  # @return [String] the result from the chat.
+  #
+  def chat_without_middlewares(messages, **params, &block)
+    @client.chat(messages, model: @model, **params, &block)
+  end
 end

data/lib/ai_client/config.yml

@@ -1,23 +1,8 @@
 ---
-:logger: !ruby/object:Logger
-  level: 0
-  progname:
-  default_formatter: !ruby/object:Logger::Formatter
-    datetime_format:
-  formatter:
-  logdev: !ruby/object:Logger::LogDevice
-    shift_period_suffix:
-    shift_size:
-    shift_age:
-    filename:
-    dev: !ruby/object:IO {}
-    binmode: false
-    reraise_write_errors: []
-    mon_data: !ruby/object:Monitor {}
-    mon_data_owner_object_id: 45380
-  level_override: {}
+:logger:
 :timeout:
 :return_raw: false
+:context_length: 5
 :providers: {}
 :envar_api_key_names:
   :anthropic:
@@ -39,3 +24,12 @@
   :localai: !ruby/regexp /^local-/i
   :ollama: !ruby/regexp /(llama|nomic)/i
   :open_router: !ruby/regexp /\//
+:default_provider: :openai
+:default_model:
+  :anthropic: claude-3-5-sonnet-20240620
+  :openai: gpt-4o
+  :google: gemini-pro-1.5
+  :mistral: mistral-large
+  :localai: llama3.2
+  :ollama: llama3.2
+  :open_router: auto

data/lib/ai_client/configuration.rb

@@ -152,9 +152,10 @@ class AiClient
     #
     def initialize_defaults
       @default_config = Config.new(
-        logger: Logger.new(STDOUT),
+        logger: nil, # Logger.new(STDOUT),
         timeout: nil,
         return_raw: false,
+        context_length: 5, # number of responses to add as context
         providers: {},
         envar_api_key_names: {
           anthropic: ['ANTHROPIC_API_KEY'],
@@ -171,6 +172,16 @@ class AiClient
           localai: /^local-/i,
           ollama: /(llama|nomic)/i,
           open_router: /\//
+        },
+        default_provider: :openai,
+        default_model: {
+          anthropic: 'claude-3-5-sonnet-20240620',
+          openai: 'gpt-4o',
+          google: 'gemini-pro-1.5',
+          mistral: 'mistral-large',
+          localai: 'llama3.2',
+          ollama: 'llama3.2',
+          open_router: 'auto'
         }
       )
 

data/lib/ai_client/llm.rb

@@ -9,6 +9,13 @@ class AiClient
     DATA_PATH = Pathname.new( __dir__ + '/models.yml')
     self.data = YAML.parse(DATA_PATH.read).to_ruby 
 
+    scope :providers,  -> {all.map(&:provider).uniq.map(&:to_sym)}
+
+    scope :models, ->(substring=nil) do
+      (substring.nil? ? all : all.where(id: /#{substring}/i))
+        .map(&:model).sort.uniq
+    end
+
     # Extracts the model name from the LLM ID.
     #
     # @return [String] the model name.
@@ -17,9 +24,12 @@ class AiClient
 
     # Extracts the provider name from the LLM ID.
     #
-    # @return [String] the provider name.
+    # @return [Symbol] the provider name.
     #
-    def provider  = id.split('/')[0]
+    def provider  = id.split('/')[0].to_sym
+  
+    def to_h = attributes
+
   end
 
   class << self
@@ -34,5 +44,6 @@ class AiClient
       AiClient::LLM.data = orc_models
       AiClient::LLM::DATA_PATH.write(orc_models.to_yaml)
     end
+
   end
 end

data/lib/ai_client/middleware.rb

@@ -1,6 +1,6 @@
 # lib/ai_client/middleware.rb
 
-# TODO: As concurrently designed the middleware must
+# TODO: As currently designed the middleware must
 #       be set before an instance of AiClient is created.
 #       Any `use` commands for middleware made after
 #       the instance is created will not be available
@@ -23,7 +23,7 @@ class AiClient
   #
   def call_with_middlewares(method, *args, **kwargs, &block)
     stack = self.class.middlewares.reverse.reduce(-> { send(method, *args, **kwargs, &block) }) do |next_middleware, middleware|
-      -> { middleware.call(self, next_middleware, *args, **kwargs) }
+      -> { middleware.call(self, next_middleware, *args, **kwargs, &block) }
     end
     stack.call
   end