langchainrb 0.9.3 → 0.9.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b31725a5fdb7c09d25e97b3b8ecbd78eba1eeece6ef2db82f009aa121e1f4956
-  data.tar.gz: 1f9116daf6780682d8c32021212bba702a053af9b4293e488ca605cc298c8e12
+  metadata.gz: 833d4dafdf55e45852261e1c86b8121fd3ed1b61766a7fb121589e6549b255e0
+  data.tar.gz: d3834e7a5d15cf1ddd45bfc2db69b73afb5cf60b6b43004a398a688b5d5932e1
 SHA512:
-  metadata.gz: 6f50889ce152ac93567951c2a854c5589f5306c8a54c852febc9d5884b3d924904beabfd076e87fefb95354dda99fbb77d179045274ff25bf9515ecee3b2d6bb
-  data.tar.gz: aee9ed10fe48eeef9dc5ba1433145823d20c55042bea7bc359ce5cad5dd4783eb68f1e84bdad79349efea17d77455b9dd6ba918a6a6d0d991e7c350887feac6d
+  metadata.gz: 98dbc07b39f956d7425c562451d9eced8162cd7d7e181ac6188d029090275f5485376db3afeb826e730c168f6d64a7cd6af90503974f8b2696b1555f3d18b589
+  data.tar.gz: 3e641d27e3ccdedfa363c7bfecb7f6a1293c1f866421db3ac5e74dcd4615934a132345f9c7912fec0200d2fe626747faaefa592592066e336d33c4db5d3fd050

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 ## [Unreleased]
 
+## [0.9.5]
+- Now using OpenAI's "text-embedding-3-small" model to generate embeddings
+- Added `remove_texts(ids:)` method to Qdrant and Chroma
+- Add Ruby 3.3 support
+
+## [0.9.4]
+- New `Ollama#summarize()` method
+- Improved README
+- Fixes + specs
+
 ## [0.9.3]
 - Add EML processor
 - Tools can support multiple-methods

data/README.md

@@ -42,7 +42,7 @@ If bundler is not being used to manage dependencies, install the gem by executin
 
     gem install langchainrb
 
-Additional gems may be required when loading LLM Providers. These are not included by default so you can include only what you need.
+Additional gems may be required. They're not included by default so you can include only what you need.
 
 ## Usage
 
@@ -51,10 +51,10 @@ require "langchain"
 ```
 
 ## Large Language Models (LLMs)
-Langchain.rb wraps all supported LLMs in a unified interface allowing you to easily swap out and test out different models.
+Langchain.rb wraps supported LLMs in a unified interface allowing you to easily swap out and test out different models.
 
 #### Supported LLMs and features:
-| LLM providers                                                                                   | embed()            | complete()         | chat()              | summarize()        | Notes              |
+| LLM providers                                                                                   | `embed()`            | `complete()`         | `chat()`              | `summarize()`        | Notes              |
 | --------                                                                                        |:------------------:| :-------:          | :-----------------: | :-------:          | :----------------- |
 | [OpenAI](https://openai.com/?utm_source=langchainrb&utm_medium=github)                          | ✅                 | ✅                 | ✅                  | ❌                 | Including Azure OpenAI |
 | [AI21](https://ai21.com/?utm_source=langchainrb&utm_medium=github)                              | ❌                 | ✅                 | ❌                  | ✅                 |                    |
@@ -64,7 +64,7 @@ Langchain.rb wraps all supported LLMs in a unified interface allowing you to eas
 | [GooglePalm](https://ai.google/discover/palm2?utm_source=langchainrb&utm_medium=github)         | ✅                 | ✅                 | ✅                  | ✅                 |                    |
 | [Google Vertex AI](https://cloud.google.com/vertex-ai?utm_source=langchainrb&utm_medium=github) | ✅                 | ✅                 | ❌                  | ✅                 |                    |
 | [HuggingFace](https://huggingface.co/?utm_source=langchainrb&utm_medium=github)                 | ✅                 | ❌                 | ❌                  | ❌                 |                    |
-| [Ollama](https://ollama.ai/?utm_source=langchainrb&utm_medium=github)                           | ✅                 | ✅                 | ✅                  | ❌                 |                    |
+| [Ollama](https://ollama.ai/?utm_source=langchainrb&utm_medium=github)                           | ✅                 | ✅                 | ✅                  | ✅                 |                    |
 | [Replicate](https://replicate.com/?utm_source=langchainrb&utm_medium=github)                    | ✅                 | ✅                 | ✅                  | ✅                 |                    |
 
 #### Using standalone LLMs:
@@ -83,12 +83,7 @@ llm = Langchain::LLM::OpenAI.new(api_key: ENV["OPENAI_API_KEY"], llm_options: {
 
 Generate vector embeddings:
 ```ruby
-llm.embed(text: "foo bar")
-```
-
-Generate a text completion:
-```ruby
-llm.complete(prompt: "What is the meaning of life?").completion
+llm.embed(text: "foo bar").embedding
 ```
 
 Generate a chat completion:
@@ -249,7 +244,7 @@ Then parse the llm response:
 
 ```ruby
 llm = Langchain::LLM::OpenAI.new(api_key: ENV["OPENAI_API_KEY"])
-llm_response = llm.chat(prompt: prompt_text).completion
+llm_response = llm.chat(messages: [{role: "user", content: prompt_text}]).completion
 parser.parse(llm_response)
 # {
 #   "name" => "Kim Ji-hyun",
@@ -398,14 +393,9 @@ client.similarity_search_by_vector(
 
 RAG-based querying
 ```ruby
-client.ask(
-  question:
-)
+client.ask(question: "...")
 ```
 
-## Evaluations (Evals)
-The Evaluations module is a collection of tools that can be used to evaluate and track the performance of the output products by LLM and your RAG (Retrieval Augmented Generation) pipelines.
-
 ## Assistants
 Assistants are Agent-like objects that leverage helpful instructions, LLMs, tools and knowledge to respond to user queries. Assistants can be configured with an LLM of your choice (currently only OpenAI), any vector search database and easily extended with additional tools.
 
@@ -473,6 +463,9 @@ assistant.thread.messages
 
 The Assistant checks the context window limits before every request to the LLM and remove oldest thread messages one by one if the context window is exceeded.
 
+## Evaluations (Evals)
+The Evaluations module is a collection of tools that can be used to evaluate and track the performance of the output products by LLM and your RAG (Retrieval Augmented Generation) pipelines.
+
 ### RAGAS
 Ragas helps you evaluate your Retrieval Augmented Generation (RAG) pipelines. The implementation is based on this [paper](https://arxiv.org/abs/2309.15217) and the original Python [repo](https://github.com/explodinggradients/ragas). Ragas tracks the following 3 metrics and assigns the 0.0 - 1.0 scores:
 * Faithfulness - the answer is grounded in the given context.
@@ -501,7 +494,7 @@ Additional examples available: [/examples](https://github.com/andreibondarev/lan
 
 ## Logging
 
-LangChain.rb uses standard logging mechanisms and defaults to `:warn` level. Most messages are at info level, but we will add debug or warn statements as needed.
+Langchain.rb uses standard logging mechanisms and defaults to `:warn` level. Most messages are at info level, but we will add debug or warn statements as needed.
 To show all log messages:
 
 ```ruby

data/lib/langchain/agent/react_agent.rb

@@ -26,6 +26,8 @@ module Langchain::Agent
     # @param max_iterations [Integer] The maximum number of iterations to run
     # @return [ReActAgent] The Agent::ReActAgent instance
     def initialize(llm:, tools: [], max_iterations: 10)
+      warn "[DEPRECATION] `Langchain::Agent::ReActAgent` is deprecated. Please use `Langchain::Assistant` instead."
+
       Langchain::Tool::Base.validate_tools!(tools: tools)
 
       @tools = tools

data/lib/langchain/agent/sql_query_agent.rb

@@ -11,6 +11,8 @@ module Langchain::Agent
     # @param db [Object] Database connection info
     #
     def initialize(llm:, db:)
+      warn "[DEPRECATION] `Langchain::Agent::ReActAgent` is deprecated. Please use `Langchain::Assistant` instead."
+
       @llm = llm
       @db = db
       @schema = @db.dump_schema

data/lib/langchain/assistants/assistant.rb

@@ -1,6 +1,8 @@
 # frozen_string_literal: true
 
 module Langchain
+  # Assistants are Agent-like objects that leverage helpful instructions, LLMs, tools and knowledge to respond to user queries.
+  # Assistants can be configured with an LLM of your choice (currently only OpenAI), any vector search database and easily extended with additional tools.
   class Assistant
     attr_reader :llm, :thread, :instructions
     attr_accessor :tools
@@ -176,26 +178,6 @@ module Langchain
       Message.new(role: role, content: content, tool_calls: tool_calls, tool_call_id: tool_call_id)
     end
 
-    # # TODO: Fix the message truncation when context window is exceeded
-    # def build_assistant_prompt(instructions:, tools:)
-    #   while begin
-    #     # Check if the prompt exceeds the context window
-    #     # Return false to exit the while loop
-    #     !llm.class.const_get(:LENGTH_VALIDATOR).validate_max_tokens!(
-    #       thread.messages,
-    #       llm.defaults[:chat_completion_model_name],
-    #       {llm: llm}
-    #     )
-    #   # Rescue error if context window is exceeded and return true to continue the while loop
-    #   rescue Langchain::Utils::TokenLength::TokenLimitExceeded
-    #     # Should be using `retry` instead of while()
-    #     true
-    #   end
-    #     # Truncate the oldest messages when the context window is exceeded
-    #     thread.messages.shift
-    #   end
-
-    #   prompt
-    # end
+    # TODO: Fix the message truncation when context window is exceeded
   end
 end

data/lib/langchain/assistants/thread.rb

@@ -1,8 +1,8 @@
 # frozen_string_literal: true
 
 module Langchain
-  # Langchain::Thread keeps track of messages in a conversation
-  # Eventually we may want to add functionality to persist to the thread to disk, DB, storage, etc.
+  # Langchain::Thread keeps track of messages in a conversation.
+  # TODO: Add functionality to persist to the thread to disk, DB, storage, etc.
   class Thread
     attr_accessor :messages
 

data/lib/langchain/chunker/markdown.rb

@@ -4,12 +4,10 @@ require "baran"
 
 module Langchain
   module Chunker
-    #
     # Simple text chunker
     #
     # Usage:
     #     Langchain::Chunker::Markdown.new(text).chunks
-    #
     class Markdown < Base
       attr_reader :text, :chunk_size, :chunk_overlap
 

data/lib/langchain/chunker/recursive_text.rb

@@ -4,12 +4,10 @@ require "baran"
 
 module Langchain
   module Chunker
-    #
     # Recursive text chunker. Preferentially splits on separators.
     #
     # Usage:
     #     Langchain::Chunker::RecursiveText.new(text).chunks
-    #
     class RecursiveText < Base
       attr_reader :text, :chunk_size, :chunk_overlap, :separators
 

data/lib/langchain/chunker/semantic.rb

@@ -2,7 +2,6 @@
 
 module Langchain
   module Chunker
-    #
     # LLM-powered semantic chunker.
     # Semantic chunking is a technique of splitting texts by their semantic meaning, e.g.: themes, topics, and ideas.
     # We use an LLM to accomplish this. The Anthropic LLM is highly recommended for this task as it has the longest context window (100k tokens).
@@ -12,7 +11,6 @@ module Langchain
     #       text,
     #       llm: Langchain::LLM::Anthropic.new(api_key: ENV["ANTHROPIC_API_KEY"])
     #     ).chunks
-    #
     class Semantic < Base
       attr_reader :text, :llm, :prompt_template
       # @param [Langchain::LLM::Base] Langchain::LLM::* instance
@@ -28,7 +26,7 @@ module Langchain
         prompt = prompt_template.format(text: text)
 
         # Replace static 50k limit with dynamic limit based on text length (max_tokens_to_sample)
-        completion = llm.complete(prompt: prompt, max_tokens_to_sample: 50000)
+        completion = llm.complete(prompt: prompt, max_tokens_to_sample: 50000).completion
         completion
           .gsub("Here are the paragraphs split by topic:\n\n", "")
           .split("---")

data/lib/langchain/chunker/sentence.rb

@@ -4,12 +4,10 @@ require "pragmatic_segmenter"
 
 module Langchain
   module Chunker
-    #
     # This chunker splits text by sentences.
     #
     # Usage:
     #     Langchain::Chunker::Sentence.new(text).chunks
-    #
     class Sentence < Base
       attr_reader :text
 

data/lib/langchain/chunker/text.rb

@@ -4,12 +4,10 @@ require "baran"
 
 module Langchain
   module Chunker
-    #
     # Simple text chunker
     #
     # Usage:
     #     Langchain::Chunker::Text.new(text).chunks
-    #
     class Text < Base
       attr_reader :text, :chunk_size, :chunk_overlap, :separator
 

data/lib/langchain/contextual_logger.rb

@@ -42,7 +42,7 @@ module Langchain
       for_class_name = for_class&.name
 
       log_line_parts = []
-      log_line_parts << "[LangChain.rb]".colorize(color: :yellow)
+      log_line_parts << "[Langchain.rb]".colorize(color: :yellow)
       log_line_parts << if for_class.respond_to?(:logger_options)
         "[#{for_class_name}]".colorize(for_class.logger_options) + ":"
       elsif for_class_name

data/lib/langchain/conversation/memory.rb

@@ -9,6 +9,8 @@ module Langchain
       TOKEN_LEEWAY = 20
 
       def initialize(llm:, messages: [], **options)
+        warn "[DEPRECATION] `Langchain::Conversation::Memory` is deprecated. Please use `Langchain::Assistant` instead."
+
         @llm = llm
         @context = nil
         @summary = nil

data/lib/langchain/conversation/message.rb

@@ -12,6 +12,8 @@ module Langchain
       }
 
       def initialize(content)
+        warn "[DEPRECATION] `Langchain::Conversation::*` is deprecated. Please use `Langchain::Assistant` and `Langchain::Messages` classes instead."
+
         @content = content
       end
 

data/lib/langchain/llm/ollama.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require "active_support/core_ext/hash"
+
 module Langchain::LLM
   # Interface to Ollama API.
   # Available models: https://ollama.ai/library
@@ -17,6 +19,16 @@ module Langchain::LLM
       chat_completion_model_name: "llama2"
     }.freeze
 
+    EMBEDDING_SIZES = {
+      codellama: 4_096,
+      "dolphin-mixtral": 4_096,
+      llama2: 4_096,
+      llava: 4_096,
+      mistral: 4_096,
+      "mistral-openorca": 4_096,
+      mixtral: 4_096
+    }.freeze
+
     # Initialize the Ollama client
     # @param url [String] The URL of the Ollama instance
     # @param default_options [Hash] The default options to use
@@ -24,7 +36,17 @@ module Langchain::LLM
     def initialize(url:, default_options: {})
       depends_on "faraday"
       @url = url
-      @defaults = DEFAULTS.merge(default_options)
+      @defaults = DEFAULTS.deep_merge(default_options)
+    end
+
+    # Returns the # of vector dimensions for the embeddings
+    # @return [Integer] The # of vector dimensions
+    def default_dimension
+      # since Ollama can run multiple models, look it up or generate an embedding and return the size
+      @default_dimension ||=
+        EMBEDDING_SIZES.fetch(defaults[:embeddings_model_name].to_sym) do
+          embed(text: "test").embedding.size
+        end
     end
 
     #
@@ -108,9 +130,11 @@ module Langchain::LLM
         req.body = parameters
 
         req.options.on_data = proc do |chunk, size|
-          json_chunk = JSON.parse(chunk)
+          chunk.split("\n").each do |line_chunk|
+            json_chunk = JSON.parse(line_chunk)
 
-          response += json_chunk.dig("response")
+            response += json_chunk.dig("response")
+          end
 
           yield json_chunk, size if block
         end
@@ -217,6 +241,19 @@ module Langchain::LLM
       Langchain::LLM::OllamaResponse.new(response.body, model: parameters[:model])
     end
 
+    # Generate a summary for a given text
+    #
+    # @param text [String] The text to generate a summary for
+    # @return [String] The summary
+    def summarize(text:)
+      prompt_template = Langchain::Prompt.load_from_path(
+        file_path: Langchain.root.join("langchain/llm/prompts/ollama/summarize_template.yaml")
+      )
+      prompt = prompt_template.format(text: text)
+
+      complete(prompt: prompt)
+    end
+
     private
 
     # @return [Faraday::Connection] Faraday client

data/lib/langchain/llm/openai.rb

@@ -9,7 +9,7 @@ module Langchain::LLM
   # Usage:
   #    openai = Langchain::LLM::OpenAI.new(
   #      api_key: ENV["OPENAI_API_KEY"],
-  #      llm_options: {},
+  #      llm_options: {}, # Available options: https://github.com/alexrudall/ruby-openai/blob/main/lib/openai/client.rb#L5-L13
   #      default_options: {}
   #    )
   class OpenAI < Base
@@ -17,8 +17,13 @@ module Langchain::LLM
       n: 1,
       temperature: 0.0,
       chat_completion_model_name: "gpt-3.5-turbo",
-      embeddings_model_name: "text-embedding-ada-002",
-      dimension: 1536
+      embeddings_model_name: "text-embedding-3-small"
+    }.freeze
+
+    EMBEDDING_SIZES = {
+      "text-embedding-ada-002": 1536,
+      "text-embedding-3-large": 3072,
+      "text-embedding-3-small": 1536
     }.freeze
 
     LENGTH_VALIDATOR = Langchain::Utils::TokenLength::OpenAIValidator
@@ -48,7 +53,8 @@ module Langchain::LLM
       text:,
       model: defaults[:embeddings_model_name],
       encoding_format: nil,
-      user: nil
+      user: nil,
+      dimensions: EMBEDDING_SIZES.fetch(model.to_sym, nil)
     )
       raise ArgumentError.new("text argument is required") if text.empty?
       raise ArgumentError.new("model argument is required") if model.empty?
@@ -61,6 +67,10 @@ module Langchain::LLM
       parameters[:encoding_format] = encoding_format if encoding_format
       parameters[:user] = user if user
 
+      if ["text-embedding-3-small", "text-embedding-3-large"].include?(model)
+        parameters[:dimensions] = EMBEDDING_SIZES[model.to_sym] if EMBEDDING_SIZES.key?(model.to_sym)
+      end
+
       validate_max_tokens(text, parameters[:model])
 
       response = with_api_error_handling do
@@ -77,6 +87,8 @@ module Langchain::LLM
     # @param params [Hash] The parameters to pass to the `chat()` method
     # @return [Langchain::LLM::OpenAIResponse] Response object
     def complete(prompt:, **params)
+      warn "DEPRECATED: `Langchain::LLM::OpenAI#complete` is deprecated, and will be removed in the next major version. Use `Langchain::LLM::OpenAI#chat` instead."
+
       if params[:stop_sequences]
         params[:stop] = params.delete(:stop_sequences)
       end
@@ -170,6 +182,10 @@ module Langchain::LLM
       complete(prompt: prompt)
     end
 
+    def default_dimension
+      @defaults[:dimension] || EMBEDDING_SIZES.fetch(defaults[:embeddings_model_name].to_sym)
+    end
+
     private
 
     attr_reader :response_chunks

data/lib/langchain/llm/prompts/ollama/summarize_template.yaml

@@ -0,0 +1,9 @@
+_type: prompt
+input_variables:
+  - text
+template: |
+  Write a concise summary of the following TEXT. Do not include the word summary, just provide the summary.
+
+  TEXT: {text}
+
+  CONCISE SUMMARY:

data/lib/langchain/output_parsers/base.rb

@@ -5,18 +5,15 @@ module Langchain::OutputParsers
   #
   # @abstract
   class Base
-    #
     # Parse the output of an LLM call.
     #
     # @param text - LLM output to parse.
     #
     # @return [Object] Parsed output.
-    #
     def parse(text:)
       raise NotImplementedError
     end
 
-    #
     # Return a string describing the format of the output.
     #
     # @return [String] Format instructions.
@@ -27,7 +24,6 @@ module Langchain::OutputParsers
     #  "foo": "bar"
     # }
     # ```
-    #
     def get_format_instructions
       raise NotImplementedError
     end

data/lib/langchain/output_parsers/output_fixing_parser.rb

@@ -6,13 +6,11 @@ module Langchain::OutputParsers
   class OutputFixingParser < Base
     attr_reader :llm, :parser, :prompt
 
-    #
     # Initializes a new instance of the class.
     #
     # @param llm [Langchain::LLM] The LLM used in the fixing process
     # @param parser [Langchain::OutputParsers] The parser originally used which resulted in parsing error
     # @param prompt [Langchain::Prompt::PromptTemplate]
-    #
     def initialize(llm:, parser:, prompt:)
       raise ArgumentError.new("llm must be an instance of Langchain::LLM got: #{llm.class}") unless llm.is_a?(Langchain::LLM::Base)
       raise ArgumentError.new("parser must be an instance of Langchain::OutputParsers got #{parser.class}") unless parser.is_a?(Langchain::OutputParsers::Base)
@@ -30,17 +28,14 @@ module Langchain::OutputParsers
       }
     end
 
-    #
     # calls get_format_instructions on the @parser
     #
     # @return [String] Instructions for how the output of a language model should be formatted
     # according to the @schema.
-    #
     def get_format_instructions
       parser.get_format_instructions
     end
 
-    #
     # Parse the output of an LLM call, if fails with OutputParserException
     # then call the LLM with a fix prompt in an attempt to get the correctly
     # formatted response
@@ -48,7 +43,6 @@ module Langchain::OutputParsers
     # @param completion [String] Text output from the LLM call
     #
     # @return [Object] object that is succesfully parsed by @parser.parse
-    #
     def parse(completion)
       parser.parse(completion)
     rescue OutputParserException => e
@@ -63,7 +57,6 @@ module Langchain::OutputParsers
       parser.parse(new_completion)
     end
 
-    #
     # Creates a new instance of the class using the given JSON::Schema.
     #
     # @param llm [Langchain::LLM] The LLM used in the fixing process
@@ -71,7 +64,6 @@ module Langchain::OutputParsers
     # @param prompt [Langchain::Prompt::PromptTemplate]
     #
     # @return [Object] A new instance of the class
-    #
     def self.from_llm(llm:, parser:, prompt: nil)
       new(llm: llm, parser: parser, prompt: prompt || naive_fix_prompt)
     end

data/lib/langchain/output_parsers/structured_output_parser.rb

@@ -5,15 +5,12 @@ require "json-schema"
 
 module Langchain::OutputParsers
   # = Structured Output Parser
-  #
   class StructuredOutputParser < Base
     attr_reader :schema
 
-    #
     # Initializes a new instance of the class.
     #
     # @param schema [JSON::Schema] The json schema
-    #
     def initialize(schema:)
       @schema = validate_schema!(schema)
     end
@@ -25,24 +22,20 @@ module Langchain::OutputParsers
       }
     end
 
-    #
     # Creates a new instance of the class using the given JSON::Schema.
     #
     # @param schema [JSON::Schema] The JSON::Schema to use
     #
     # @return [Object] A new instance of the class
-    #
     def self.from_json_schema(schema)
       new(schema: schema)
     end
 
-    #
     # Returns a string containing instructions for how the output of a language model should be formatted
     # according to the @schema.
     #
     # @return [String] Instructions for how the output of a language model should be formatted
     # according to the @schema.
-    #
     def get_format_instructions
       <<~INSTRUCTIONS
         You must format your output as a JSON value that adheres to a given "JSON Schema" instance.
@@ -62,13 +55,10 @@ module Langchain::OutputParsers
       INSTRUCTIONS
     end
 
-    #
     # Parse the output of an LLM call extracting an object that abides by the @schema
     #
     # @param text [String] Text output from the LLM call
-    #
     # @return [Object] object that abides by the @schema
-    #
     def parse(text)
       json = text.include?("```") ? text.strip.split(/```(?:json)?/)[1] : text.strip
       parsed = JSON.parse(json)

data/lib/langchain/processors/eml.rb

@@ -1,4 +1,3 @@
-require "mail"
 require "uri"
 
 module Langchain

data/lib/langchain/tool/ruby_code_interpreter/ruby_code_interpreter.rb

@@ -1,41 +1,45 @@
 # frozen_string_literal: true
 
-module Langchain::Tool
-  class RubyCodeInterpreter < Base
-    #
-    # A tool that execute Ruby code in a sandboxed environment.
-    #
-    # Gem requirements:
-    #     gem "safe_ruby", "~> 1.0.4"
-    #
-    # Usage:
-    #    interpreter = Langchain::Tool::RubyCodeInterpreter.new
-    #
-    NAME = "ruby_code_interpreter"
-    ANNOTATIONS_PATH = Langchain.root.join("./langchain/tool/#{NAME}/#{NAME}.json").to_path
+# RubyCodeInterpreter does not work with Ruby 3.3;
+# https://github.com/ukutaht/safe_ruby/issues/4
+if RUBY_VERSION <= "3.2"
+  module Langchain::Tool
+    class RubyCodeInterpreter < Base
+      #
+      # A tool that execute Ruby code in a sandboxed environment.
+      #
+      # Gem requirements:
+      #     gem "safe_ruby", "~> 1.0.4"
+      #
+      # Usage:
+      #    interpreter = Langchain::Tool::RubyCodeInterpreter.new
+      #
+      NAME = "ruby_code_interpreter"
+      ANNOTATIONS_PATH = Langchain.root.join("./langchain/tool/#{NAME}/#{NAME}.json").to_path
 
-    description <<~DESC
-      A Ruby code interpreter. Use this to execute ruby expressions. Input should be a valid ruby expression. If you want to see the output of the tool, make sure to return a value.
-    DESC
+      description <<~DESC
+        A Ruby code interpreter. Use this to execute ruby expressions. Input should be a valid ruby expression. If you want to see the output of the tool, make sure to return a value.
+      DESC
 
-    def initialize(timeout: 30)
-      depends_on "safe_ruby"
+      def initialize(timeout: 30)
+        depends_on "safe_ruby"
 
-      @timeout = timeout
-    end
+        @timeout = timeout
+      end
 
-    # Executes Ruby code in a sandboxes environment.
-    #
-    # @param input [String] ruby code expression
-    # @return [String] Answer
-    def execute(input:)
-      Langchain.logger.info("Executing \"#{input}\"", for: self.class)
+      # Executes Ruby code in a sandboxes environment.
+      #
+      # @param input [String] ruby code expression
+      # @return [String] Answer
+      def execute(input:)
+        Langchain.logger.info("Executing \"#{input}\"", for: self.class)
 
-      safe_eval(input)
-    end
+        safe_eval(input)
+      end
 
-    def safe_eval(code)
-      SafeRuby.eval(code, timeout: @timeout)
+      def safe_eval(code)
+        SafeRuby.eval(code, timeout: @timeout)
+      end
     end
   end
 end

data/lib/langchain/vectorsearch/base.rb

@@ -136,7 +136,7 @@ module Langchain::Vectorsearch
     # @param k [Integer] The number of results to return
     # @return [String] Response
     def similarity_search_with_hyde(query:, k: 4)
-      hyde_completion = llm.complete(prompt: generate_hyde_prompt(question: query))
+      hyde_completion = llm.complete(prompt: generate_hyde_prompt(question: query)).completion
       similarity_search(query: hyde_completion, k: k)
     end
 

data/lib/langchain/vectorsearch/chroma.rb

@@ -60,6 +60,13 @@ module Langchain::Vectorsearch
       collection.update(embeddings)
     end
 
+    # Remove a list of texts from the index
+    # @param ids [Array<String>] The list of ids to remove
+    # @return [Hash] The response from the server
+    def remove_texts(ids:)
+      collection.delete(ids)
+    end
+
     # Create the collection with the default schema
     # @return [::Chroma::Resources::Collection] Created collection
     def create_default_schema

data/lib/langchain/vectorsearch/qdrant.rb

@@ -64,6 +64,16 @@ module Langchain::Vectorsearch
       add_texts(texts: texts, ids: ids)
     end
 
+    # Remove a list of texts from the index
+    # @param ids [Array<Integer>] The ids to remove
+    # @return [Hash] The response from the server
+    def remove_texts(ids:)
+      client.points.delete(
+        collection_name: index_name,
+        points: ids
+      )
+    end
+
     # Get the default schema
     # @return [Hash] The response from the server
     def get_default_schema

data/lib/langchain/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Langchain
-  VERSION = "0.9.3"
+  VERSION = "0.9.5"
 end

data/lib/langchain.rb

@@ -72,7 +72,7 @@ loader.setup
 #
 # = Logging
 #
-# LangChain.rb uses standard logging mechanisms and defaults to :debug level. Most messages are at info level, but we will add debug or warn statements as needed. To show all log messages:
+# Langchain.rb uses standard logging mechanisms and defaults to :debug level. Most messages are at info level, but we will add debug or warn statements as needed. To show all log messages:
 #
 # Langchain.logger.level = :info
 module Langchain

metadata

@@ -1,15 +1,29 @@
 --- !ruby/object:Gem::Specification
 name: langchainrb
 version: !ruby/object:Gem::Version
-  version: 0.9.3
+  version: 0.9.5
 platform: ruby
 authors:
 - Andrei Bondarev
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2024-02-23 00:00:00.000000000 Z
+date: 2024-03-15 00:00:00.000000000 Z
 dependencies:
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 7.0.8
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 7.0.8
 - !ruby/object:Gem::Dependency
   name: baran
   requirement: !ruby/object:Gem::Requirement
@@ -178,6 +192,34 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 2.2.7
+- !ruby/object:Gem::Dependency
+  name: vcr
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: ai21
   requirement: !ruby/object:Gem::Requirement
@@ -626,7 +668,7 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description: Build LLM-backed Ruby applications with Ruby's LangChain
+description: Build LLM-backed Ruby applications with Ruby's Langchain.rb
 email:
 - andrei.bondarev13@gmail.com
 executables: []
@@ -684,6 +726,7 @@ files:
 - lib/langchain/llm/llama_cpp.rb
 - lib/langchain/llm/ollama.rb
 - lib/langchain/llm/openai.rb
+- lib/langchain/llm/prompts/ollama/summarize_template.yaml
 - lib/langchain/llm/prompts/summarize_template.yaml
 - lib/langchain/llm/replicate.rb
 - lib/langchain/llm/response/ai21_response.rb
@@ -776,8 +819,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.1
+rubygems_version: 3.5.3
 signing_key:
 specification_version: 4
-summary: Build LLM-backed Ruby applications with Ruby's LangChain
+summary: Build LLM-backed Ruby applications with Ruby's Langchain.rb
 test_files: []