aoororachain 0.1.3 → 0.1.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 55f23c15305124c7290243efeb8d0af00c2db4d39a86a53f33b385ac7f5c0eea
-  data.tar.gz: 69554775d50528239ef2405fb79f0349ae7ec56bd915992ae088daf1f33ca080
+  metadata.gz: 5f3616ebd00d6c06568a9b1c5d31a99147eb5f0afe48749f92e8275ee31c8272
+  data.tar.gz: 27711347c6c7306a16dace3cc74d931b2585fbfe953ef61166e7a51b38893dca
 SHA512:
-  metadata.gz: a126b07255d4b2b4ebd06017d351e0ab7ef220713503bcdd8b897dc02ebac1bf0810fffd52250a63cfd5396397b1c5711d25a8779da825e437eb677dca498249
-  data.tar.gz: 80624eb9468969821af3fecf082b5f161216e7140f9b23a7ae170b33569b5c1618c8b256c41a1febe2c9ef5a9dba62e937cd025f4dcd579f9edd78e574090521
+  metadata.gz: 29096f58726afdfdc68f78f6effc57fcc8e79e645d2fccd51c87ca560d4f9ffed90c66873e9d9e69e795ec882ff6b02f12d951e5d4fd4406a9b7a11e64e56615
+  data.tar.gz: 7a67196e23436c158aa259cc2cb5c6e8c6e8cad6a5548ecbdfdd35cd9ca019b0d9e09378272140d9f893479be51fd48dd0f09bc956fe87a57d39345ea591b606

data/CHANGELOG.md

@@ -1,5 +1,8 @@
 ## [Unreleased]
 
-## [0.1.0] - 2023-05-24
+## [0.1.5] - 2023-07-10
+- Implements QA Retrieval with additional context.
+
+## [0.1.0] - 2023-06-24
 
 - Initial release

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    aoororachain (0.1.3)
+    aoororachain (0.1.5)
       chroma-db (~> 0.6.0)
       llm_client (~> 0.1.2)
       pdf-reader (~> 2.11)

data/README.md

@@ -1,26 +1,263 @@
 # Aoororachain
 
-Aoororachain is Ruby chain tool to work with LLMs
+Aoororachain is Ruby chain tool to work with LLMs.
 
 ## Installation
 
-Install the gem and add to the application's Gemfile by executing:
+Install the gem and add to the application’s Gemfile by executing:
 
-    $ bundle add aoororachain
+```bash
+$ bundle add aoororachain
+```
 
 If bundler is not being used to manage dependencies, install the gem by executing:
 
-    $ gem install aoororachain
+```bash
+$ gem install aoororachain
+```
+
+## Requisites
+
+Aoororachain was primarily created to work locally with private data and Open Source LLMs. If you are looking for a tool to integrate OpenAI or any other service, there are a handful of tools to do it in Ruby, Python, or Javascript in Github.
+
+With this in mind, a few requisites are needed before you start working in a chain.
+
+* Llama.cpp. First, you need to setup [llama.cpp](https://github.com/ggerganov/llama.cpp), an inference tool for the Llama model.
+* LLM Server. [LLM Server](https://github.com/mariochavez/llm_server) is a Ruby server that exposes *llama.cpp* via an API interfase.
+* Open Source LLM model. Refer to *llama.cpp* or *LLM Server* for options to download an Open Source model. Llama, Open Llama or Vicuna models are good models to start.
+* Chroma DB. [Chroma DB]( [https://www.trychroma.com/](https://www.trychroma.com/) ) is an Open Source Vector database for document information retrieval.
+* Python environment. Aoororachain uses Open Source embedding models. It uses by default any of `hkunlp/instructor-large`, `hkunlp/instructor-xl`, and `sentence-transformers/all-mpnet-base-v2`.
+
+### Python environment and Open Source embedding models.
+
+You can install a Python environment using [miniconda](https://docs.conda.io/en/latest/miniconda.html). Here are the instructions for using it and installing additional dependencies and the Embedding models.
+
+```bash
+# This assumes installing miniconda on MacOS with Homebrew. If you use a different OS, follow the instructions on miniconda website.
+$ brew install miniconda
+# Initialize miniconda with your shell. Restart your shell for this to take effect.
+$ conda init zsh
+# After the shell restarts, create an environment and set Python version.
+$ conda create -n llm python=3.9
+# Now activate your new environment
+$ conda activate llm
+# Install Embedding models dependencies
+$ pip -q install langchain sentence_transformers InstructorEmbedding
+```
+
+The next step is to install the Embedding model or models you want to use. Here are the links to each model.
+
+* [hkunlp/instructor-xl](https://huggingface.co/hkunlp/instructor-xl). 5Gb.
+* [hkunlp/instructor-large](https://huggingface.co/hkunlp/instructor-large). 1.34Gb
+* [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2). 438Mb
+
+To install any models, execute the following code in a Python repl. Replace *MODEL* with the name of the model. _Be aware that this will download the model from Internet._
+
+```python
+from InstructorEmbedding import INSTRUCTOR
+          from langchain.embeddings
+import HuggingFaceInstructEmbeddings
+
+instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="MODEL")
+
+instructor_embeddings.embed_documents(list("Hello Ruby!"))
+```
+
+You can skip this step, but Aoororachain will download the specified model on the first run.
 
 ## Usage
 
-TODO: Write usage instructions here
+Aoororachain currently focused on QA Retrieval for your own documents. Hence, let's start with how to create embeddings for a set of documents.
+
+### Document embeddings
+
+Being able to QA your documents requires texts to be converted to numbers. These numbers are organized in vectors; they capture the word features and correlations in sentences. This is helpful when a question is asked and, through the vector, a program can find texts that are similar to the question asked.
+
+The similar texts can then be sent to a Large Language Model (LLM) to make sense of them and produce a response in Natural Language Process (NLP).
+
+Due to the context size limit of LLMs you can feed them a huge document for QA Retrieval, you need to chunk large texts into meaningful blocks. This process is part of the embedding creation process.
+
+The process looks like the following: 
+
+1. Load documents—in this example, Ruby 3.2 documentation from 9,747 text files.
+
+This is an example of one of the 9,747 text files:
+
+```ruby
+Object Array
+Method collect
+Method type instance_method
+Call sequence ["array.map {|element| ... } -> new_array\narray.map -> new_enumerator"]
+Source code 3.2:ruby-3.2.0/array.c:3825
+
+Calls the block, if given, with each element of self; returns a new Array whose elements are the return values from the block:
+
+a = [:foo, 'bar', 2]
+a1 = a.map {|element| element.class }
+a1 # => [Symbol, String, Integer]
+
+Returns a new Enumerator if no block given:
+a = [:foo, 'bar', 2]
+a1 = a.map
+a1 # => #
+
+Array#collect is an alias for Array#map.
+Examples static VALUE
+rb_ary_collect(VALUE ary)
+{
+long i;
+VALUE collect;
+
+RETURN_SIZED_ENUMERATOR(ary, 0, 0, ary_enum_length);
+collect = rb_ary_new2(RARRAY_LEN(ary));
+for (i = 0; i < RARRAY_LEN(ary); i++) {
+  rb_ary_push(collect, rb_yield(RARRAY_AREF(ary, i)));
+}
+return collect;
+}
+```
+
+2. Chunk texts into meaningful blocks.
+3. Create embeddings for texts.
+4. Store embeddings in a vector database.
+
+Aoororachain uses the Chroma vector database to store and query embeddings.
+
+Here is an example for loading and creating the embeddings. 
+
+```ruby
+require "aoororachain"
+
+# Setup logger.
+Aoororachain.logger = Logger.new($stdout)
+Aoororachain.log_level = Aoororachain::LEVEL_DEBUG
+
+chroma_host = "http://localhost:8000"
+collection_name = "ruby-documentation"
+
+# You can define a custom Parser to clean data and maybe extract metadata.
+# Here is the code of RubyDocParser that does exactly that.
+class RubyDocParser
+  def self.parse(text)
+    name_match = text.match(/Name (\w+)/)
+    constant_match = text.match(/Constant (\w+)/)
+    
+    object_match = text.match(/Object (\w+)/)
+    method_match = text.match(/Method ([\w\[\]\+\=\-\*\%\/]+)/)
+    
+    metadata = {}
+    metadata[:name] = name_match[1] if name_match
+    metadata[:constant] = constant_match[1] if constant_match
+    metadata[:object] = object_match[1] if object_match
+    metadata[:method] = method_match[1] if method_match
+    metadata[:lang] = :ruby
+    metadata[:version] = "3.2"
+    
+    text.gsub!(/\s+/, " ").strip!
+    [text, metadata]
+  end
+end
+
+# A DirectoryLoader points to a path and sets the glob for the files you want to load. 
+# A loader is also specified. FileLoader just opens and reads the file content. 
+# The RubyDocParser is set as well. This is optional in case you data is very nice and needs no pre-processing.
+directory_loader = Aoororachain::Loaders::DirectoryLoader.new(path: "./ruby-docs", glob: "**/*.txt", loader: Aoororachain::Loaders::FileLoader, parser: RubyDocParser)
+files = directory_loader.load
+
+# With your data clean and ready, now it is time to chunk it. The chunk size depends of the context size of the LLMs that you want to use.
+# 512 is a good number to start, don't go lower than that. An overlap can also be specified.
+text_splitter = Aoororachain::RecursiveTextSplitter.new(size: 512, overlap: 0)
+
+texts = []
+files.each do |file|
+  texts.concat(text_splitter.split_documents(file))
+end
+
+# The final step is to create and store the embeddings.
+# First, select an embedding model
+model = Aoororachain::Embeddings::LocalPythonEmbedding::MODEL_INSTRUCTOR_L
+# Create an instance of the embedder. device is optional. Possible options are:
+# - cuda. If you have an external GPU
+# - mps. If you have an Apple Sillicon chip (M1 to M2).
+# - cpu or empty. It will use the CPU by default.
+embedder = Aoororachain::Embeddings::LocalPythonEmbedding.new(model:, device: "mps")
+# Configure your Vector database.
+vector_database = Aoororachain::VectorStores::Chroma.new(embedder: embedder, options: {host: chroma_host})
+
+# Embbed your files. This can take a few minutes up to hours, depending on the size of your documents and the model used.
+vector_database.from_documents(texts, index: collection_name)
+```
+
+With embedding loaded in the database, you can use a tool like Chroma UI -**not yet released** - to query documents.
+![chroma-ui](https://github.com/mariochavez/aoororachain/assets/59967/d65dea13-c6ef-452a-9774-8cf3b47c048f)
+
+Now you can query your embeddings with Aoororachain.
+
+```ruby
+# Define a retriever for the Vector database.
+retriever = Aoororachain::VectorStores::Retriever.new(vector_database)
+
+# Query documents, results by default is 3.
+documents = retriever.search("how can I use the Data class?", results: 4)
+
+# Print retrieved documents and their similarity distance from the question.
+puts documents.map(&:document).join(" ")
+puts documents.map(&:distance)
+```
+
+### Query LLM with context.
+
+With embeddings ready, it is time to create a _chain_ to perform QA Retrieval using the embedded documents as context.
+
+```ruby
+require "aoororachain"
+
+# Setup logger.
+Aoororachain.logger = Logger.new($stdout)
+Aoororachain.log_level = Aoororachain::LEVEL_DEBUG
+
+llm_host = "http://localhost:9292"
+chroma_host = "http://localhost:8000"
+collection_name = "ruby-documentation"
+
+model = Aoororachain::Embeddings::LocalPythonEmbedding::MODEL_INSTRUCTOR_L
+embedder = Aoororachain::Embeddings::LocalPythonEmbedding.new(model:, device: "mps")
+vector_database = Aoororachain::VectorStores::Chroma.new(embedder: embedder, options: {host: chroma_host, log_level: Chroma::LEVEL_DEBUG})
+vector_database.from_index(collection_name)
+
+retriever = Aoororachain::VectorStores::Retriever.new(vector_database)
+
+# Configure the LLM Server
+llm = Aoororachain::Llms::LlamaServer.new(llm_host)
+
+# Create the chain to connect the Vector database retriever with the LLM.
+chain = Aoororachain::Chains::RetrievalQA.new(llm, retriever)
+
+# Create a template for the LLM. Aoororachain does not include any templates because these are model specific. The following template is for the Vicuna model.
+template = "A conversation between a human and an AI assistant. The assistant responds to a question using the context. Context: ===%{context}===. Question: %{prompt}"
+
+response = chain.complete(prompt: "given the following array [[1,3], [2,4]], how can I get a flatten and sorted array?", prompt_template: template)
+```
 
-## Development
+_response_ is a Hash with two keys: _response_ and _sources_.
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+```ruby
+pp response
+{:response=>
+  "User: Assistant: Assistant: To flatten the nested arrays in an array and sort it, you can use Ruby's built-in `sort` method along with the `flatten` method. Here is an example of how to do this for the given array [[1, 3], [2, 4]]:\n" +
+  "```ruby\n" +
+  "array = [[1, 3], [2, 4]]\n" +
+  "sorted_and_flattened_array = array.sort { |a, b| a[0] <=> b[0] }.flat_map(&:to_a)\n" +
+  "# Output: [1, 2, 3, 4]\n" +
+  "```\n",
+ :sources=>
+  [{"source"=>"./ruby-docs/hash-flatten.txt", "object"=>"Hash", "method"=>"flatten", "lang"=>"ruby", "version"=>"3.2"},
+   {"source"=>"./ruby-docs/array-flatten.txt", "object"=>"Array", "method"=>"flatten", "lang"=>"ruby", "version"=>"3.2"},
+   {"source"=>"./ruby-docs/array-flatten.txt", "object"=>"Array", "method"=>"flatten", "lang"=>"ruby", "version"=>"3.2"},
+   {"source"=>"./ruby-docs/array-flatten2.txt", "object"=>"Array", "method"=>"flatten", "lang"=>"ruby", "version"=>"3.2"}]}
+```
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+Where _response_ is tge generated response from the LLM and _sources_ is the list of text chunks that were sent to the LLM as context.
 
 ## Contributing
 

data/lib/aoororachain/chains/retrieval_qa.rb

@@ -9,25 +9,25 @@ module Aoororachain
         @type = type
       end
 
-      def complete(prompt:)
+      def complete(prompt:, prompt_template:, additional_context: "")
         context = @retriever.search(prompt)
 
-        system_prompt = "Una conversación entre un humano y un asistente de inteligencia artificial. El asistente response usando el contexto la pregunta. Si no sabes la respuesta, simplemente di que no sabes, no trates de inventar una."
-        context_prompt = "Contexto: #{context.map(&:document).join(" ").tr("\n", " ")}"
-        question_prompt = "Pregunta: #{prompt}"
+        mapped_context = context.map(&:document)
+        mapped_context << additional_context if !additional_context.nil? || additional_context != ""
 
-        stuff_prompt = [system_prompt, context_prompt, question_prompt]
-        success, response = @llm.complete(prompt: stuff_prompt.join(". "))
+        stuff_prompt = prompt_template % {context: mapped_context.join(" ").tr("\n", " "), prompt:}
+
+        success, response = @llm.complete(prompt: stuff_prompt)
 
         if success
           completion = {
-            response: response,
-            sources: context.map(&:metadata)
-          }
+            "sources" => context.map(&:metadata)
+          }.merge(response)
         else
           completion = {
-            response: "Sorry we had a problem with the LLM",
-            sources: []
+            "response" => "Sorry we had a problem with the LLM",
+            "sources" => [],
+            "model" => ""
           }
           Aoororachain::Util.log_error("Failed to complete", message: response)
         end

data/lib/aoororachain/embeddings/local_python_embedding.rb

@@ -12,9 +12,7 @@ module Aoororachain
         @device = options.delete(:device) || "cpu"
 
         Aoororachain::Util.log_info("Using", data: {model: @model, device: @device})
-        Aoororachain::Util.log_info("This embedding calls Python code using system call. First time initialization might take long due to Python dependencies installation.")
-
-        install_python_dependencies
+        Aoororachain::Util.log_info("This embedding calls Python code using system call.")
       end
 
       def embed_documents(documents, include_metadata: false)
@@ -151,18 +149,6 @@ module Aoororachain
 
         file_path
       end
-
-      def install_python_dependencies
-        stdout_data, stderr_data, exit_code = run_system("pip -q install langchain sentence_transformers InstructorEmbedding")
-
-        if exit_code != 0
-          Aoororachain.log_error("Failed to install Python dependencies: #{stderr_data}")
-          return false
-        end
-
-        Aoororachain::Util.log_debug("Python installed dependencies: #{stdout_data}")
-        true
-      end
     end
   end
 end

data/lib/aoororachain/llms/llama_server.rb

@@ -14,7 +14,7 @@ module Aoororachain
       def complete(prompt:)
         result = LlmClient.completion(prompt)
 
-        [result.success?, result.success? ? result.success.body["response"].gsub(/Usuario:.*Asistente:/, "") : result.failure.message]
+        [result.success?, result.success? ? result.success.body : result.failure.body]
       end
     end
   end

data/lib/aoororachain/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Aoororachain
-  VERSION = "0.1.3"
+  VERSION = "0.1.5"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: aoororachain
 version: !ruby/object:Gem::Version
-  version: 0.1.3
+  version: 0.1.5
 platform: ruby
 authors:
 - Mario Alberto Chávez
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2023-06-27 00:00:00.000000000 Z
+date: 2023-07-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: chroma-db
@@ -106,7 +106,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.14
+rubygems_version: 3.4.15
 signing_key:
 specification_version: 4
 summary: Aoororachain for working with LLMs