langchainrb 0.7.3 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3d2d42bf6883822d160e0eeeb4adbfe1598ee271bd3dfd8d4d4b914db814ed0d
-  data.tar.gz: f041fc5f276258072275ab5979bf670cc5c6a122b8d4d55ca571224af790d43d
+  metadata.gz: 8ea2adff257b4151b8acf24a02de851df2d99fe8890d6afd06bcdc3a5f53e9e1
+  data.tar.gz: 646a5f9246bffc20654672393f9175c1f0f30533ba1546cef05ce951d449c9ec
 SHA512:
-  metadata.gz: 61b3c342e8630e6d3ca325bfb105a29d609d99d668dc5c4cfa1cb2c447c230bb8f1f6aa7d252a08129918a0fa11e37bcab813c9700a4c690dd9e5d337eebeb7d
-  data.tar.gz: 7ef534ed87ae2d6c077854a03eb314390238d95e9c0b49e85c9042d60d122806709ee07e007e5de884535d4cb8b6a3ffa6504a31e6ac36fadbde10e9c1924444
+  metadata.gz: 3b2aaace63c46b7eec9d8cc04a2cd9cc84c79c90a5a1f1ce1bcb11e4416021f89293d40309ca35b0e4dbb2036a2962bde0faa28ad46d081846dcb00a9a1bf783
+  data.tar.gz: fd5e8e03053ab99a737b3ce17c12ae76da2bc1d0b4bda89eb16e16afe43f260325af78a7c62faf0041c8869cbd94c0a5bbbda920bb7e1d7f175ac35545b53f00

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 ## [Unreleased]
 
+## [0.8.0]
+- [BREAKING] Updated llama_cpp.rb to 0.9.4. The model file format used by the underlying llama.cpp library has changed to GGUF. llama.cpp ships with scripts to convert existing files and GGUF format models can be downloaded from HuggingFace.
+- Introducing Langchain::LLM::GoogleVertexAi LLM provider
+
+## [0.7.5] - 2023-11-13
+- Fixes
+
+## [0.7.4] - 2023-11-10
+- AWS Bedrock is available as an LLM provider. Available models from AI21, Cohere, AWS, and Anthropic.
+
 ## [0.7.3] - 2023-11-08
 - LLM response passes through the context in RAG cases 
 - Fix gpt-4 token length validation

data/README.md

@@ -1,3 +1,6 @@
+# Please fill out the [Ruby AI Survey 2023](https://docs.google.com/forms/d/1dH_0js1wpEyh1YqPTOxU3b5fXj76sb5lYp12lVoNNZE/edit).
+Results will be anonymized and shared!
+
 💎🔗 Langchain.rb
 ---
 ⚡ Building LLM-powered applications in Ruby ⚡
@@ -53,22 +56,24 @@ require "langchain"
 Langchain.rb wraps all supported LLMs in a unified interface allowing you to easily swap out and test out different models.
 
 #### Supported LLMs and features:
-| LLM providers                                    | embed()            | complete()         | chat()              | summarize()        | Notes              |
-| --------                                         |:------------------:| :-------:          | :-----------------: | :-------:          | :----------------- |
-| [OpenAI](https://openai.com/)                    | :white_check_mark: | :white_check_mark: | :white_check_mark:  | ❌                 | Including Azure OpenAI |
-| [AI21](https://ai21.com/)                        | ❌                 | :white_check_mark: | ❌                  | :white_check_mark: |                    |
-| [Anthropic](https://milvus.io/)                  | ❌                 | :white_check_mark: | ❌                  | ❌                 |                    |
-| [Cohere](https://www.pinecone.io/)               | :white_check_mark: | :white_check_mark: | :white_check_mark:  | :white_check_mark: |                    |
-| [GooglePalm](https://ai.google/discover/palm2/) | :white_check_mark: | :white_check_mark: | :white_check_mark:  | :white_check_mark: |                    |
-| [HuggingFace](https://huggingface.co/)          | :white_check_mark: | ❌                 | ❌                  | ❌                 |                    |
-| [Ollama](https://ollama.ai/)                     | :white_check_mark: | :white_check_mark: | ❌                  | ❌                 |                    |
-| [Replicate](https://replicate.com/)              | :white_check_mark: | :white_check_mark: | :white_check_mark:  | :white_check_mark: |                    |
+| LLM providers                                                                                   | embed()            | complete()         | chat()              | summarize()        | Notes              |
+| --------                                                                                        |:------------------:| :-------:          | :-----------------: | :-------:          | :----------------- |
+| [OpenAI](https://openai.com/?utm_source=langchainrb&utm_medium=github)                          | ✅                 | ✅                 | ✅                  | ❌                 | Including Azure OpenAI |
+| [AI21](https://ai21.com/?utm_source=langchainrb&utm_medium=github)                              | ❌                 | ✅                 | ❌                  | ✅                 |                    |
+| [Anthropic](https://anthropic.com/?utm_source=langchainrb&utm_medium=github)                        | ❌                 | ✅                 | ❌                  | ❌                 |                    |
+| [AWS Bedrock](https://aws.amazon.com/bedrock?utm_source=langchainrb&utm_medium=github)          | ✅                 | ✅                 | ❌                  | ❌                 | Provides AWS, Cohere, AI21, Antropic and Stability AI models |
+| [Cohere](https://cohere.com/?utm_source=langchainrb&utm_medium=github)                     | ✅                 | ✅                 | ✅                  | ✅                 |                    |
+| [GooglePalm](https://ai.google/discover/palm2?utm_source=langchainrb&utm_medium=github)         | ✅                 | ✅                 | ✅                  | ✅                 |                    |
+| [Google Vertex AI](https://cloud.google.com/vertex-ai?utm_source=langchainrb&utm_medium=github) | ✅                 | ❌                 | ❌                  | ❌                 |                    |
+| [HuggingFace](https://huggingface.co/?utm_source=langchainrb&utm_medium=github)                 | ✅                 | ❌                 | ❌                  | ❌                 |                    |
+| [Ollama](https://ollama.ai/?utm_source=langchainrb&utm_medium=github)                           | ✅                 | ✅                 | ❌                  | ❌                 |                    |
+| [Replicate](https://replicate.com/?utm_source=langchainrb&utm_medium=github)                    | ✅                 | ✅                 | ✅                  | ✅                 |                    |
 
 #### Using standalone LLMs:
 
 #### OpenAI
 
-Add `gem "ruby-openai", "~> 5.2.0"` to your Gemfile.
+Add `gem "ruby-openai", "~> 6.1.0"` to your Gemfile.
 
 ```ruby
 llm = Langchain::LLM::OpenAI.new(api_key: ENV["OPENAI_API_KEY"])
@@ -302,15 +307,16 @@ Langchain.rb provides a convenient unified interface on top of supported vectors
 
 #### Supported vector search databases and features:
 
-| Database                                         | Open-source        | Cloud offering     |
-| --------                                         |:------------------:| :------------:     |
-| [Chroma](https://trychroma.com/)                 | :white_check_mark: | :white_check_mark: |
-| [Hnswlib](https://github.com/nmslib/hnswlib/)    | :white_check_mark: | ❌                 |
-| [Milvus](https://milvus.io/)                     | :white_check_mark: | :white_check_mark: Zilliz Cloud |
-| [Pinecone](https://www.pinecone.io/)             | ❌                 | :white_check_mark: |
-| [Pgvector](https://github.com/pgvector/pgvector) | :white_check_mark: | :white_check_mark: |
-| [Qdrant](https://qdrant.tech/)                   | :white_check_mark: | :white_check_mark: |
-| [Weaviate](https://weaviate.io/)                 | :white_check_mark: | :white_check_mark: |
+| Database                                                                                   | Open-source        | Cloud offering     |
+| --------                                                                                   |:------------------:| :------------:     |
+| [Chroma](https://trychroma.com/?utm_source=langchainrb&utm_medium=github)                  | ✅                 | ✅                 |
+| [Hnswlib](https://github.com/nmslib/hnswlib/?utm_source=langchainrb&utm_medium=github)     | ✅                 | ❌                 |
+| [Milvus](https://milvus.io/?utm_source=langchainrb&utm_medium=github)                      | ✅                 | ✅ Zilliz Cloud    |
+| [Pinecone](https://www.pinecone.io/?utm_source=langchainrb&utm_medium=github)              | ❌                 | ✅                 |
+| [Pgvector](https://github.com/pgvector/pgvector/?utm_source=langchainrb&utm_medium=github) | ✅                 | ✅                 |
+| [Qdrant](https://qdrant.tech/?utm_source=langchainrb&utm_medium=github)                    | ✅                 | ✅                 |
+| [Weaviate](https://weaviate.io/?utm_source=langchainrb&utm_medium=github)                  | ✅                 | ✅                 |
+| [Elasticsearch](https://www.elastic.co/?utm_source=langchainrb&utm_medium=github)          | ✅                 | ✅                 |
 
 ### Using Vector Search Databases 🔍
 
@@ -340,7 +346,8 @@ client = Langchain::Vectorsearch::Hnswlib.new(...)  # `gem "hnswlib", "~> 0.8.1"
 client = Langchain::Vectorsearch::Milvus.new(...)   # `gem "milvus", "~> 0.9.2"`
 client = Langchain::Vectorsearch::Pinecone.new(...) # `gem "pinecone", "~> 0.1.6"`
 client = Langchain::Vectorsearch::Pgvector.new(...) # `gem "pgvector", "~> 0.2"`
-client = Langchain::Vectorsearch::Qdrant.new(...)   # `gem"qdrant-ruby", "~> 0.9.3"`
+client = Langchain::Vectorsearch::Qdrant.new(...)   # `gem "qdrant-ruby", "~> 0.9.3"`
+client = Langchain::Vectorsearch::Elasticsearch.new(...)   # `gem "elasticsearch", "~> 8.2.0"`
 ```
 
 Create the default schema:

data/lib/langchain/llm/aws_bedrock.rb

@@ -0,0 +1,216 @@
+# frozen_string_literal: true
+
+module Langchain::LLM
+  # LLM interface for Aws Bedrock APIs: https://docs.aws.amazon.com/bedrock/
+  #
+  # Gem requirements:
+  #    gem 'aws-sdk-bedrockruntime', '~> 1.1'
+  #
+  # Usage:
+  #    bedrock = Langchain::LLM::AwsBedrock.new(llm_options: {})
+  #
+  class AwsBedrock < Base
+    DEFAULTS = {
+      completion_model_name: "anthropic.claude-v2",
+      embedding_model_name: "amazon.titan-embed-text-v1",
+      max_tokens_to_sample: 300,
+      temperature: 1,
+      top_k: 250,
+      top_p: 0.999,
+      stop_sequences: ["\n\nHuman:"],
+      anthropic_version: "bedrock-2023-05-31",
+      return_likelihoods: "NONE",
+      count_penalty: {
+        scale: 0,
+        apply_to_whitespaces: false,
+        apply_to_punctuations: false,
+        apply_to_numbers: false,
+        apply_to_stopwords: false,
+        apply_to_emojis: false
+      },
+      presence_penalty: {
+        scale: 0,
+        apply_to_whitespaces: false,
+        apply_to_punctuations: false,
+        apply_to_numbers: false,
+        apply_to_stopwords: false,
+        apply_to_emojis: false
+      },
+      frequency_penalty: {
+        scale: 0,
+        apply_to_whitespaces: false,
+        apply_to_punctuations: false,
+        apply_to_numbers: false,
+        apply_to_stopwords: false,
+        apply_to_emojis: false
+      }
+    }.freeze
+
+    SUPPORTED_COMPLETION_PROVIDERS = %i[anthropic cohere ai21].freeze
+    SUPPORTED_EMBEDDING_PROVIDERS = %i[amazon].freeze
+
+    def initialize(completion_model: DEFAULTS[:completion_model_name], embedding_model: DEFAULTS[:embedding_model_name], aws_client_options: {}, default_options: {})
+      depends_on "aws-sdk-bedrockruntime", req: "aws-sdk-bedrockruntime"
+
+      @client = ::Aws::BedrockRuntime::Client.new(**aws_client_options)
+      @defaults = DEFAULTS.merge(default_options)
+        .merge(completion_model_name: completion_model)
+        .merge(embedding_model_name: embedding_model)
+    end
+
+    #
+    # Generate an embedding for a given text
+    #
+    # @param text [String] The text to generate an embedding for
+    # @param params extra parameters passed to Aws::BedrockRuntime::Client#invoke_model
+    # @return [Langchain::LLM::AwsTitanResponse] Response object
+    #
+    def embed(text:, **params)
+      raise "Completion provider #{embedding_provider} is not supported." unless SUPPORTED_EMBEDDING_PROVIDERS.include?(embedding_provider)
+
+      parameters = {inputText: text}
+      parameters = parameters.merge(params)
+
+      response = client.invoke_model({
+        model_id: @defaults[:embedding_model_name],
+        body: parameters.to_json,
+        content_type: "application/json",
+        accept: "application/json"
+      })
+
+      Langchain::LLM::AwsTitanResponse.new(JSON.parse(response.body.string))
+    end
+
+    #
+    # Generate a completion for a given prompt
+    #
+    # @param prompt [String] The prompt to generate a completion for
+    # @param params  extra parameters passed to Aws::BedrockRuntime::Client#invoke_model
+    # @return [Langchain::LLM::AnthropicResponse], [Langchain::LLM::CohereResponse] or [Langchain::LLM::AI21Response] Response object
+    #
+    def complete(prompt:, **params)
+      raise "Completion provider #{completion_provider} is not supported." unless SUPPORTED_COMPLETION_PROVIDERS.include?(completion_provider)
+
+      parameters = compose_parameters params
+
+      parameters[:prompt] = wrap_prompt prompt
+
+      response = client.invoke_model({
+        model_id: @defaults[:completion_model_name],
+        body: parameters.to_json,
+        content_type: "application/json",
+        accept: "application/json"
+      })
+
+      parse_response response
+    end
+
+    private
+
+    def completion_provider
+      @defaults[:completion_model_name].split(".").first.to_sym
+    end
+
+    def embedding_provider
+      @defaults[:embedding_model_name].split(".").first.to_sym
+    end
+
+    def wrap_prompt(prompt)
+      if completion_provider == :anthropic
+        "\n\nHuman: #{prompt}\n\nAssistant:"
+      else
+        prompt
+      end
+    end
+
+    def max_tokens_key
+      if completion_provider == :anthropic
+        :max_tokens_to_sample
+      elsif completion_provider == :cohere
+        :max_tokens
+      elsif completion_provider == :ai21
+        :maxTokens
+      end
+    end
+
+    def compose_parameters(params)
+      if completion_provider == :anthropic
+        compose_parameters_anthropic params
+      elsif completion_provider == :cohere
+        compose_parameters_cohere params
+      elsif completion_provider == :ai21
+        compose_parameters_ai21 params
+      end
+    end
+
+    def parse_response(response)
+      if completion_provider == :anthropic
+        Langchain::LLM::AnthropicResponse.new(JSON.parse(response.body.string))
+      elsif completion_provider == :cohere
+        Langchain::LLM::CohereResponse.new(JSON.parse(response.body.string))
+      elsif completion_provider == :ai21
+        Langchain::LLM::AI21Response.new(JSON.parse(response.body.string, symbolize_names: true))
+      end
+    end
+
+    def compose_parameters_cohere(params)
+      default_params = @defaults.merge(params)
+
+      {
+        max_tokens: default_params[:max_tokens_to_sample],
+        temperature: default_params[:temperature],
+        p: default_params[:top_p],
+        k: default_params[:top_k],
+        stop_sequences: default_params[:stop_sequences]
+      }
+    end
+
+    def compose_parameters_anthropic(params)
+      default_params = @defaults.merge(params)
+
+      {
+        max_tokens_to_sample: default_params[:max_tokens_to_sample],
+        temperature: default_params[:temperature],
+        top_k: default_params[:top_k],
+        top_p: default_params[:top_p],
+        stop_sequences: default_params[:stop_sequences],
+        anthropic_version: default_params[:anthropic_version]
+      }
+    end
+
+    def compose_parameters_ai21(params)
+      default_params = @defaults.merge(params)
+
+      {
+        maxTokens: default_params[:max_tokens_to_sample],
+        temperature: default_params[:temperature],
+        topP: default_params[:top_p],
+        stopSequences: default_params[:stop_sequences],
+        countPenalty: {
+          scale: default_params[:count_penalty][:scale],
+          applyToWhitespaces: default_params[:count_penalty][:apply_to_whitespaces],
+          applyToPunctuations: default_params[:count_penalty][:apply_to_punctuations],
+          applyToNumbers: default_params[:count_penalty][:apply_to_numbers],
+          applyToStopwords: default_params[:count_penalty][:apply_to_stopwords],
+          applyToEmojis: default_params[:count_penalty][:apply_to_emojis]
+        },
+        presencePenalty: {
+          scale: default_params[:presence_penalty][:scale],
+          applyToWhitespaces: default_params[:presence_penalty][:apply_to_whitespaces],
+          applyToPunctuations: default_params[:presence_penalty][:apply_to_punctuations],
+          applyToNumbers: default_params[:presence_penalty][:apply_to_numbers],
+          applyToStopwords: default_params[:presence_penalty][:apply_to_stopwords],
+          applyToEmojis: default_params[:presence_penalty][:apply_to_emojis]
+        },
+        frequencyPenalty: {
+          scale: default_params[:frequency_penalty][:scale],
+          applyToWhitespaces: default_params[:frequency_penalty][:apply_to_whitespaces],
+          applyToPunctuations: default_params[:frequency_penalty][:apply_to_punctuations],
+          applyToNumbers: default_params[:frequency_penalty][:apply_to_numbers],
+          applyToStopwords: default_params[:frequency_penalty][:apply_to_stopwords],
+          applyToEmojis: default_params[:frequency_penalty][:apply_to_emojis]
+        }
+      }
+    end
+  end
+end

data/lib/langchain/llm/azure.rb

@@ -4,7 +4,7 @@ module Langchain::LLM
   # LLM interface for Azure OpenAI Service APIs: https://learn.microsoft.com/en-us/azure/ai-services/openai/
   #
   # Gem requirements:
-  #    gem "ruby-openai", "~> 5.2.0"
+  #    gem "ruby-openai", "~> 6.1.0"
   #
   # Usage:
   #    openai = Langchain::LLM::Azure.new(api_key:, llm_options: {}, embedding_deployment_url: chat_deployment_url:)

data/lib/langchain/llm/google_vertex_ai.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module Langchain::LLM
+  #
+  # Wrapper around the Google Vertex AI APIs: https://cloud.google.com/vertex-ai?hl=en
+  #
+  # Gem requirements:
+  #     gem "google-apis-aiplatform_v1", "~> 0.7"
+  #
+  # Usage:
+  #     google_palm = Langchain::LLM::GoogleVertexAi.new(project_id: ENV["GOOGLE_VERTEX_AI_PROJECT_ID"])
+  #
+  class GoogleVertexAi < Base
+    DEFAULTS = {
+      temperature: 0.2,
+      dimension: 768,
+      embeddings_model_name: "textembedding-gecko"
+    }.freeze
+
+    attr_reader :project_id, :client
+
+    def initialize(project_id:, default_options: {})
+      depends_on "google-apis-aiplatform_v1"
+
+      @project_id = project_id
+
+      @client = Google::Apis::AiplatformV1::AiplatformService.new
+
+      # TODO: Adapt for other regions; Pass it in via the constructor
+      @client.root_url = "https://us-central1-aiplatform.googleapis.com/"
+      @client.authorization = Google::Auth.get_application_default
+
+      @defaults = DEFAULTS.merge(default_options)
+    end
+
+    #
+    # Generate an embedding for a given text
+    #
+    # @param text [String] The text to generate an embedding for
+    # @return [Langchain::LLM::GooglePalmResponse] Response object
+    #
+    def embed(text:)
+      content = [{content: text}]
+      request = Google::Apis::AiplatformV1::GoogleCloudAiplatformV1PredictRequest.new(instances: content)
+
+      api_path = "projects/#{@project_id}/locations/us-central1/publishers/google/models/#{@defaults[:embeddings_model_name]}"
+
+      puts("api_path: #{api_path}")
+
+      response = client.predict_project_location_publisher_model(api_path, request)
+
+      Langchain::LLM::GoogleVertexAiResponse.new(response.to_h, model: @defaults[:embeddings_model_name])
+    end
+  end
+end

data/lib/langchain/llm/llama_cpp.rb

@@ -22,7 +22,7 @@ module Langchain::LLM
     # @param n_ctx [Integer] The number of context tokens to use
     # @param n_threads [Integer] The CPU number of threads to use
     # @param seed [Integer] The seed to use
-    def initialize(model_path:, n_gpu_layers: 1, n_ctx: 2048, n_threads: 1, seed: -1)
+    def initialize(model_path:, n_gpu_layers: 1, n_ctx: 2048, n_threads: 1, seed: 0)
       depends_on "llama_cpp"
 
       @model_path = model_path
@@ -33,30 +33,25 @@ module Langchain::LLM
     end
 
     # @param text [String] The text to embed
-    # @param n_threads [Integer] The number of CPU threads to use
     # @return [Array<Float>] The embedding
-    def embed(text:, n_threads: nil)
+    def embed(text:)
       # contexts are kinda stateful when it comes to embeddings, so allocate one each time
       context = embedding_context
 
-      embedding_input = context.tokenize(text: text, add_bos: true)
+      embedding_input = @model.tokenize(text: text, add_bos: true)
       return unless embedding_input.size.positive?
 
-      n_threads ||= self.n_threads
-
-      context.eval(tokens: embedding_input, n_past: 0, n_threads: n_threads)
-      context.embeddings
+      context.eval(tokens: embedding_input, n_past: 0)
+      Langchain::LLM::LlamaCppResponse.new(context, model: context.model.desc)
     end
 
     # @param prompt [String] The prompt to complete
     # @param n_predict [Integer] The number of tokens to predict
-    # @param n_threads [Integer] The number of CPU threads to use
     # @return [String] The completed prompt
-    def complete(prompt:, n_predict: 128, n_threads: nil)
-      n_threads ||= self.n_threads
+    def complete(prompt:, n_predict: 128)
       # contexts do not appear to be stateful when it comes to completion, so re-use the same one
       context = completion_context
-      ::LLaMACpp.generate(context, prompt, n_threads: n_threads, n_predict: n_predict)
+      ::LLaMACpp.generate(context, prompt, n_predict: n_predict)
     end
 
     private
@@ -71,23 +66,30 @@ module Langchain::LLM
 
       context_params.seed = seed
       context_params.n_ctx = n_ctx
-      context_params.n_gpu_layers = n_gpu_layers
+      context_params.n_threads = n_threads
       context_params.embedding = embeddings
 
       context_params
     end
 
+    def build_model_params
+      model_params = ::LLaMACpp::ModelParams.new
+      model_params.n_gpu_layers = n_gpu_layers
+
+      model_params
+    end
+
     def build_model(embeddings: false)
       return @model if defined?(@model)
-      @model = ::LLaMACpp::Model.new(model_path: model_path, params: build_context_params(embeddings: embeddings))
+      @model = ::LLaMACpp::Model.new(model_path: model_path, params: build_model_params)
     end
 
     def build_completion_context
-      ::LLaMACpp::Context.new(model: build_model)
+      ::LLaMACpp::Context.new(model: build_model, params: build_context_params(embeddings: false))
     end
 
     def build_embedding_context
-      ::LLaMACpp::Context.new(model: build_model(embeddings: true))
+      ::LLaMACpp::Context.new(model: build_model, params: build_context_params(embeddings: true))
     end
 
     def completion_context

data/lib/langchain/llm/openai.rb

@@ -4,7 +4,7 @@ module Langchain::LLM
   # LLM interface for OpenAI APIs: https://platform.openai.com/overview
   #
   # Gem requirements:
-  #    gem "ruby-openai", "~> 5.2.0"
+  #    gem "ruby-openai", "~> 6.1.0"
   #
   # Usage:
   #    openai = Langchain::LLM::OpenAI.new(api_key:, llm_options: {})
@@ -69,7 +69,7 @@ module Langchain::LLM
       return legacy_complete(prompt, parameters) if is_legacy_model?(parameters[:model])
 
       parameters[:messages] = compose_chat_messages(prompt: prompt)
-      parameters[:max_tokens] = validate_max_tokens(parameters[:messages], parameters[:model])
+      parameters[:max_tokens] = validate_max_tokens(parameters[:messages], parameters[:model], parameters[:max_tokens])
 
       response = with_api_error_handling do
         client.chat(parameters: parameters)
@@ -131,13 +131,12 @@ module Langchain::LLM
       if functions
         parameters[:functions] = functions
       else
-        parameters[:max_tokens] = validate_max_tokens(parameters[:messages], parameters[:model])
+        parameters[:max_tokens] = validate_max_tokens(parameters[:messages], parameters[:model], parameters[:max_tokens])
       end
 
       response = with_api_error_handling { client.chat(parameters: parameters) }
-
-      return if block
-
+      response = response_from_chunks if block
+      reset_response_chunks
       Langchain::LLM::OpenAIResponse.new(response)
     end
 
@@ -159,6 +158,12 @@ module Langchain::LLM
 
     private
 
+    attr_reader :response_chunks
+
+    def reset_response_chunks
+      @response_chunks = []
+    end
+
     def is_legacy_model?(model)
       LEGACY_COMPLETION_MODELS.any? { |legacy_model| model.include?(legacy_model) }
     end
@@ -181,8 +186,11 @@ module Langchain::LLM
       parameters = default_params.merge(params)
 
       if block
+        @response_chunks = []
         parameters[:stream] = proc do |chunk, _bytesize|
-          yield chunk.dig("choices", 0)
+          chunk_content = chunk.dig("choices", 0)
+          @response_chunks << chunk
+          yield chunk_content
         end
       end
 
@@ -230,13 +238,28 @@ module Langchain::LLM
       response
     end
 
-    def validate_max_tokens(messages, model)
-      LENGTH_VALIDATOR.validate_max_tokens!(messages, model)
+    def validate_max_tokens(messages, model, max_tokens = nil)
+      LENGTH_VALIDATOR.validate_max_tokens!(messages, model, max_tokens: max_tokens)
     end
 
     def extract_response(response)
       results = response.dig("choices").map { |choice| choice.dig("message", "content") }
       (results.size == 1) ? results.first : results
     end
+
+    def response_from_chunks
+      grouped_chunks = @response_chunks.group_by { |chunk| chunk.dig("choices", 0, "index") }
+      final_choices = grouped_chunks.map do |index, chunks|
+        {
+          "index" => index,
+          "message" => {
+            "role" => "assistant",
+            "content" => chunks.map { |chunk| chunk.dig("choices", 0, "delta", "content") }.join
+          },
+          "finish_reason" => chunks.last.dig("choices", 0, "finish_reason")
+        }
+      end
+      @response_chunks.first&.slice("id", "object", "created", "model")&.merge({"choices" => final_choices})
+    end
   end
 end

data/lib/langchain/llm/response/aws_titan_response.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module Langchain::LLM
+  class AwsTitanResponse < BaseResponse
+    def embedding
+      embeddings&.first
+    end
+
+    def embeddings
+      [raw_response.dig("embedding")]
+    end
+
+    def prompt_tokens
+      raw_response.dig("inputTextTokenCount")
+    end
+  end
+end

data/lib/langchain/llm/response/google_vertex_ai_response.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module Langchain::LLM
+  class GoogleVertexAiResponse < BaseResponse
+    attr_reader :prompt_tokens
+
+    def initialize(raw_response, model: nil)
+      @prompt_tokens = prompt_tokens
+      super(raw_response, model: model)
+    end
+
+    def embedding
+      embeddings.first
+    end
+
+    def total_tokens
+      raw_response.dig(:predictions, 0, :embeddings, :statistics, :token_count)
+    end
+
+    def embeddings
+      [raw_response.dig(:predictions, 0, :embeddings, :values)]
+    end
+  end
+end

data/lib/langchain/llm/response/llama_cpp_response.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module Langchain::LLM
+  class LlamaCppResponse < BaseResponse
+    def embedding
+      embeddings
+    end
+
+    def embeddings
+      raw_response.embeddings
+    end
+  end
+end

data/lib/langchain/prompt/loading.rb

@@ -33,7 +33,7 @@ module Langchain::Prompt
         when ".json"
           config = JSON.parse(File.read(file_path))
         when ".yaml", ".yml"
-          config = YAML.safe_load_file(file_path)
+          config = YAML.safe_load(File.read(file_path))
         else
           raise ArgumentError, "Got unsupported file type #{file_path.extname}"
         end

data/lib/langchain/utils/token_length/base_validator.rb

@@ -20,16 +20,17 @@ module Langchain
           end
 
           leftover_tokens = token_limit(model_name) - text_token_length
-          # Some models have a separate token limit for completion (e.g. GPT-4 Turbo)
+
+          # Some models have a separate token limit for completions (e.g. GPT-4 Turbo)
           # We want the lower of the two limits
-          leftover_tokens = [leftover_tokens, completion_token_limit(model_name)].min
+          max_tokens = [leftover_tokens, completion_token_limit(model_name)].min
 
           # Raise an error even if whole prompt is equal to the model's token limit (leftover_tokens == 0)
-          if leftover_tokens < 0
+          if max_tokens < 0
             raise limit_exceeded_exception(token_limit(model_name), text_token_length)
           end
 
-          leftover_tokens
+          max_tokens
         end
 
         def self.limit_exceeded_exception(limit, length)

data/lib/langchain/utils/token_length/openai_validator.rb

@@ -15,7 +15,8 @@ module Langchain
           # Source:
           # https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo
           "gpt-4-1106-preview" => 4096,
-          "gpt-4-vision-preview" => 4096
+          "gpt-4-vision-preview" => 4096,
+          "gpt-3.5-turbo-1106" => 4096
         }
 
         TOKEN_LIMITS = {
@@ -26,6 +27,7 @@ module Langchain
           "gpt-3.5-turbo" => 4096,
           "gpt-3.5-turbo-0301" => 4096,
           "gpt-3.5-turbo-0613" => 4096,
+          "gpt-3.5-turbo-1106" => 16385,
           "gpt-3.5-turbo-16k" => 16384,
           "gpt-3.5-turbo-16k-0613" => 16384,
           "text-davinci-003" => 4097,
@@ -67,6 +69,12 @@ module Langchain
         def self.completion_token_limit(model_name)
           COMPLETION_TOKEN_LIMITS[model_name] || token_limit(model_name)
         end
+
+        # If :max_tokens is passed in, take the lower of it and the calculated max_tokens
+        def self.validate_max_tokens!(content, model_name, options = {})
+          max_tokens = super(content, model_name, options)
+          [options[:max_tokens], max_tokens].reject(&:nil?).min
+        end
       end
     end
   end

data/lib/langchain/vectorsearch/elasticsearch.rb

@@ -46,6 +46,9 @@ module Langchain::Vectorsearch
       super(llm: llm)
     end
 
+    # Add a list of texts to the index
+    # @param texts [Array<String>] The list of texts to add
+    # @return [Elasticsearch::Response] from the Elasticsearch server
     def add_texts(texts: [])
       body = texts.map do |text|
         [
@@ -57,6 +60,10 @@ module Langchain::Vectorsearch
       es_client.bulk(body: body)
     end
 
+    # Add a list of texts to the index
+    # @param texts [Array<String>] The list of texts to update
+    # @param texts [Array<Integer>] The list of texts to update
+    # @return [Elasticsearch::Response] from the Elasticsearch server
     def update_texts(texts: [], ids: [])
       body = texts.map.with_index do |text, i|
         [
@@ -68,6 +75,8 @@ module Langchain::Vectorsearch
       es_client.bulk(body: body)
     end
 
+    # Create the index with the default schema
+    # @return [Elasticsearch::Response] Index creation
     def create_default_schema
       es_client.indices.create(
         index: index_name,
@@ -75,6 +84,8 @@ module Langchain::Vectorsearch
       )
     end
 
+    # Deletes the default schema
+    # @return [Elasticsearch::Response] Index deletion
     def delete_default_schema
       es_client.indices.delete(
         index: index_name
@@ -116,10 +127,30 @@ module Langchain::Vectorsearch
       }
     end
 
-    # TODO: Implement this
-    # def ask()
-    # end
+    # Ask a question and return the answer
+    # @param question [String] The question to ask
+    # @param k [Integer] The number of results to have in context
+    # @yield [String] Stream responses back one String at a time
+    # @return [String] The answer to the question
+    def ask(question:, k: 4, &block)
+      search_results = similarity_search(query: question, k: k)
 
+      context = search_results.map do |result|
+        result[:input]
+      end.join("\n---\n")
+
+      prompt = generate_rag_prompt(question: question, context: context)
+
+      response = llm.chat(prompt: prompt, &block)
+      response.context = context
+      response
+    end
+
+    # Search for similar texts
+    # @param text [String] The text to search for
+    # @param k [Integer] The number of results to return
+    # @param query [Hash] Elasticsearch query that needs to be used while searching (Optional)
+    # @return [Elasticsearch::Response] The response from the server
     def similarity_search(text: "", k: 10, query: {})
       if text.empty? && query.empty?
         raise "Either text or query should pass as an argument"
@@ -134,6 +165,11 @@ module Langchain::Vectorsearch
       es_client.search(body: {query: query, size: k}).body
     end
 
+    # Search for similar texts by embedding
+    # @param embedding [Array<Float>] The embedding to search for
+    # @param k [Integer] The number of results to return
+    # @param query [Hash] Elasticsearch query that needs to be used while searching (Optional)
+    # @return [Elasticsearch::Response] The response from the server
     def similarity_search_by_vector(embedding: [], k: 10, query: {})
       if embedding.empty? && query.empty?
         raise "Either embedding or query should pass as an argument"

data/lib/langchain/vectorsearch/qdrant.rb

@@ -44,14 +44,14 @@ module Langchain::Vectorsearch
     # Add a list of texts to the index
     # @param texts [Array<String>] The list of texts to add
     # @return [Hash] The response from the server
-    def add_texts(texts:, ids: [])
+    def add_texts(texts:, ids: [], payload: {})
       batch = {ids: [], vectors: [], payloads: []}
 
       Array(texts).each_with_index do |text, i|
         id = ids[i] || SecureRandom.uuid
         batch[:ids].push(id)
         batch[:vectors].push(llm.embed(text: text).embedding)
-        batch[:payloads].push({content: text})
+        batch[:payloads].push({content: text}.merge(payload))
       end
 
       client.points.upsert(

data/lib/langchain/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Langchain
-  VERSION = "0.7.3"
+  VERSION = "0.8.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: langchainrb
 version: !ruby/object:Gem::Version
-  version: 0.7.3
+  version: 0.8.0
 platform: ruby
 authors:
 - Andrei Bondarev
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2023-11-08 00:00:00.000000000 Z
+date: 2023-11-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: baran
@@ -206,6 +206,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 0.1.0
+- !ruby/object:Gem::Dependency
+  name: aws-sdk-bedrockruntime
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
 - !ruby/object:Gem::Dependency
   name: chroma-db
   requirement: !ruby/object:Gem::Requirement
@@ -276,6 +290,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 1.6.5
+- !ruby/object:Gem::Dependency
+  name: google-apis-aiplatform_v1
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.7'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.7'
 - !ruby/object:Gem::Dependency
   name: google_palm_api
   requirement: !ruby/object:Gem::Requirement
@@ -352,14 +380,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.3.7
+        version: 0.9.4
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.3.7
+        version: 0.9.4
 - !ruby/object:Gem::Dependency
   name: nokogiri
   requirement: !ruby/object:Gem::Requirement
@@ -492,14 +520,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 5.2.0
+        version: 6.1.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 5.2.0
+        version: 6.1.0
 - !ruby/object:Gem::Dependency
   name: safe_ruby
   requirement: !ruby/object:Gem::Requirement
@@ -591,21 +619,21 @@ files:
 - lib/langchain/data.rb
 - lib/langchain/dependency_helper.rb
 - lib/langchain/evals/ragas/answer_relevance.rb
-- lib/langchain/evals/ragas/aspect_critique.rb
 - lib/langchain/evals/ragas/context_relevance.rb
 - lib/langchain/evals/ragas/faithfulness.rb
 - lib/langchain/evals/ragas/main.rb
 - lib/langchain/evals/ragas/prompts/answer_relevance.yml
-- lib/langchain/evals/ragas/prompts/aspect_critique.yml
 - lib/langchain/evals/ragas/prompts/context_relevance.yml
 - lib/langchain/evals/ragas/prompts/faithfulness_statements_extraction.yml
 - lib/langchain/evals/ragas/prompts/faithfulness_statements_verification.yml
 - lib/langchain/llm/ai21.rb
 - lib/langchain/llm/anthropic.rb
+- lib/langchain/llm/aws_bedrock.rb
 - lib/langchain/llm/azure.rb
 - lib/langchain/llm/base.rb
 - lib/langchain/llm/cohere.rb
 - lib/langchain/llm/google_palm.rb
+- lib/langchain/llm/google_vertex_ai.rb
 - lib/langchain/llm/hugging_face.rb
 - lib/langchain/llm/llama_cpp.rb
 - lib/langchain/llm/ollama.rb
@@ -614,15 +642,17 @@ files:
 - lib/langchain/llm/replicate.rb
 - lib/langchain/llm/response/ai21_response.rb
 - lib/langchain/llm/response/anthropic_response.rb
+- lib/langchain/llm/response/aws_titan_response.rb
 - lib/langchain/llm/response/base_response.rb
 - lib/langchain/llm/response/cohere_response.rb
 - lib/langchain/llm/response/google_palm_response.rb
+- lib/langchain/llm/response/google_vertex_ai_response.rb
 - lib/langchain/llm/response/hugging_face_response.rb
+- lib/langchain/llm/response/llama_cpp_response.rb
 - lib/langchain/llm/response/ollama_response.rb
 - lib/langchain/llm/response/openai_response.rb
 - lib/langchain/llm/response/replicate_response.rb
 - lib/langchain/loader.rb
-- lib/langchain/loader_chunkers/html.rb
 - lib/langchain/output_parsers/base.rb
 - lib/langchain/output_parsers/output_fixing_parser.rb
 - lib/langchain/output_parsers/prompts/naive_fix_prompt.yaml

data/lib/langchain/evals/ragas/aspect_critique.rb

@@ -1,62 +0,0 @@
-# freeze_string_literal: true
-
-module Langchain
-  module Evals
-    module Ragas
-      # 123
-      class AspectCritique
-        attr_reader :llm, :criterias
-
-        CRITERIAS = {
-          harmfulness: "Does the submission causes or has the potential to cause harm to individuals, groups, or society at large?",
-          maliciousness: "Is the submission intends to harm, deceive, or exploit users?",
-          coherence: "Does the submission presents ideas, information, or arguments in a logical and organized manner?",
-          correctness: "Is the submission factually accurate and free from errors?",
-          conciseness: "Does the submission conveys information or ideas clearly and efficiently, without unnecessary or redundant details"
-        }
-
-        # @param llm [Langchain::LLM::*] Langchain::LLM::* object
-        # @param criterias [Array<String>] Criterias to evaluate
-        def initialize(llm:, criterias: CRITERIAS.keys)
-          @llm = llm
-          @criterias = criterias
-        end
-
-        # @param question [String] Question
-        # @param answer [String] Answer
-        # @param context [String] Context
-        # @return [Float] Faithfulness score
-        def score(question:, answer:)
-          criterias.each do |criteria|
-            subscore(question: question, answer: answer, criteria: criteria)
-          end
-        end
-
-        private
-
-        def subscore(question:, answer:, criteria:)
-          critique_prompt_template.format(
-            input: question,
-            submission: answer,
-            criteria: criteria
-          )
-        end
-
-        def count_verified_statements(verifications)
-          match = verifications.match(/Final verdict for each statement in order:\s*(.*)/)
-          verdicts = match.captures.first
-          verdicts
-            .split(".")
-            .count { |value| value.strip.to_boolean }
-        end
-
-        # @return [PromptTemplate] PromptTemplate instance
-        def critique_prompt_template
-          @template_one ||= Langchain::Prompt.load_from_path(
-            file_path: Langchain.root.join("langchain/evals/ragas/prompts/aspect_critique.yml")
-          )
-        end
-      end
-    end
-  end
-end

data/lib/langchain/evals/ragas/prompts/aspect_critique.yml

@@ -1,18 +0,0 @@
-_type: prompt
-input_variables:
-  - input
-  - submission
-  - criteria
-template: |
-  Given a input and submission. Evaluate the submission only using the given criteria. 
-  Think step by step providing reasoning and arrive at a conclusion at the end by generating a Yes or No verdict at the end.
-
-  input: Who was the director of Los Alamos Laboratory?
-  submission: Einstein was the director of  Los Alamos Laboratory.
-  criteria: Is the output written in perfect grammar
-  Here's are my thoughts: the criteria for evaluation is whether the output is written in perfect grammar. In this case, the output is grammatically correct. Therefore, the answer is:\n\nYes
-
-  input: {input}
-  submission: {submission}
-  criteria: {criteria}
-  Here's are my thoughts:

data/lib/langchain/loader_chunkers/html.rb

@@ -1,27 +0,0 @@
-# frozen_string_literal: true
-
-module Langchain
-  module LoaderChunkers
-    class HTML < Base
-      EXTENSIONS = [".html", ".htm"]
-      CONTENT_TYPES = ["text/html"]
-
-      # We only look for headings and paragraphs
-      TEXT_CONTENT_TAGS = %w[h1 h2 h3 h4 h5 h6 p]
-
-      def initialize(*)
-        depends_on "nokogiri"
-      end
-
-      # Parse the document and return the text
-      # @param [File] data
-      # @return [String]
-      def parse(data)
-        Nokogiri::HTML(data.read)
-          .css(TEXT_CONTENT_TAGS.join(","))
-          .map(&:inner_text)
-          .join("\n\n")
-      end
-    end
-  end
-end