aoororachain 0.1.2 → 0.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 36bca18450c77bf66c93e5fc700e6f9a437d5bff8a11b3864f5b892f1b88efea
-  data.tar.gz: 062d7a96f13117e213d5364220c7a8bd65383a97bbc4a72f8d408f2d05b91548
+  metadata.gz: 8fd799856d56a712154c7c6adf5c20e73e34ce7740aa70490dc84c37f7dd7905
+  data.tar.gz: e4d4a828187141c420cecff71b38c17fdd9556b9f486a21c336ee83a1b6b20d4
 SHA512:
-  metadata.gz: ae1a1b654d3b5a70c55198593cd6d81b305327b5b557338ef31379df0353d55800df4025c98fdce7f0495ee11dca6079c93b30c943b99ee14b39009f32284579
-  data.tar.gz: 2ca658e40c4053fcdee6dbdcee307ec8a9442a9e1648f852dfe75151e11e72e029ded4db7c4b2cf57e04d02618b22c094087514a542bf284caa8c6cca4a93020
+  metadata.gz: a157218dc07395105e4f6bda26f2a97df95d882b514a800401fb7a86596d9fe8583575395f4d16797f91800913c9b4fd639826001d5d1360c104fa0896da48e7
+  data.tar.gz: 554a6f6c5bbac2d094a0d35fa296feb6d5bd7fc999d6579d3d7d44c8161f52acc282b098bdcc6d387a3e5bcd74d2ae29a7602ce04509aa49948308a618d4052e

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    aoororachain (0.1.1)
+    aoororachain (0.1.4)
       chroma-db (~> 0.6.0)
       llm_client (~> 0.1.2)
       pdf-reader (~> 2.11)

data/README.md

@@ -1,26 +1,243 @@
 # Aoororachain
 
-Aoororachain is Ruby chain tool to work with LLMs
+Aoororachain is Ruby chain tool to work with LLMs.
 
 ## Installation
 
-Install the gem and add to the application's Gemfile by executing:
+Install the gem and add to the application’s Gemfile by executing:
 
-    $ bundle add aoororachain
+```bash
+$ bundle add aoororachain
+```
 
 If bundler is not being used to manage dependencies, install the gem by executing:
 
-    $ gem install aoororachain
+```bash
+$ gem install aoororachain
+```
+
+## Requisites
+
+Aoororachain was primarily created to work locally with private data and Open Source LLMs. If you are looking for a tool to integrate OpenAI or any other service, there are a handful of tools to do it in Ruby, Python, or Javascript in Github.
+
+With this in mind, a few requisites are needed before you start working in a chain.
+
+* Llama.cpp. First, you need to setup [llama.cpp](https://github.com/ggerganov/llama.cpp), an inference tool for the Llama model.
+* LLM Server. [LLM Server](https://github.com/mariochavez/llm_server) is a Ruby server that exposes *llama.cpp* via an API interfase.
+* Open Source LLM model. Refer to *llama.cpp* or *LLM Server* for options to download an Open Source model. Llama, Open Llama or Vicuna models are good models to start.
+* Chroma DB. [Chroma DB]( [https://www.trychroma.com/](https://www.trychroma.com/) ) is an Open Source Vector database for document information retrieval.
+* Python environment. Aoororachain uses Open Source embedding models. It uses by default any of `hkunlp/instructor-large`, `hkunlp/instructor-xl`, and `sentence-transformers/all-mpnet-base-v2`.
+
+### Python environment and Open Source embedding models.
+
+You can install a Python environment using [miniconda](https://docs.conda.io/en/latest/miniconda.html). Here are the instructions for using it and installing additional dependencies and the Embedding models.
+
+```bash
+# This assumes installing miniconda on MacOS with Homebrew. If you use a different OS, follow the instructions on miniconda website.
+$ brew install miniconda
+# Initialize miniconda with your shell. Restart your shell for this to take effect.
+$ conda init zsh
+# After the shell restarts, create an environment and set Python version.
+$ conda create -n llm python=3.9
+# Now activate your new environment
+$ conda activate llm
+# Install Embedding models dependencies
+$ pip -q install langchain sentence_transformers InstructorEmbedding
+```
+
+The next step is to install the Embedding model or models you want to use. Here are the links to each model.
+
+* [hkunlp/instructor-xl](https://huggingface.co/hkunlp/instructor-xl). 5Gb.
+* [hkunlp/instructor-large](https://huggingface.co/hkunlp/instructor-large). 1.34Gb
+* [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2). 438Mb
+
+To install any models, execute the following code in a Python repl. Replace *MODEL* with the name of the model. _Be aware that this will download the model from Internet._
+
+```python
+from InstructorEmbedding import INSTRUCTOR
+          from langchain.embeddings
+import HuggingFaceInstructEmbeddings
+
+instructor_embeddings = HuggingFaceInstructEmbeddings(model_name="MODEL")
+
+instructor_embeddings.embed_documents(list("Hello Ruby!"))
+```
+
+You can skip this step, but Aoororachain will download the specified model on the first run.
 
 ## Usage
 
-TODO: Write usage instructions here
+Aoororachain currently focused on QA Retrieval for your own documents. Hence, let's start with how to create embeddings for a set of documents.
+
+### Document embeddings
+
+Being able to QA your documents requires texts to be converted to numbers. These numbers are organized in vectors; they capture the word features and correlations in sentences. This is helpful when a question is asked and, through the vector, a program can find texts that are similar to the question asked.
+
+The similar texts can then be sent to a Large Language Model (LLM) to make sense of them and produce a response in Natural Language Process (NLP).
+
+Due to the context size limit of LLMs you can feed them a huge document for QA Retrieval, you need to chunk large texts into meaningful blocks. This process is part of the embedding creation process.
+
+The process looks like the following: 
+
+1. Load documents—in this example, Ruby 3.2 documentation from 9,747 text files.
+
+This is an example of one of the 9,747 text files:
+
+```ruby
+Object Array
+Method collect
+Method type instance_method
+Call sequence ["array.map {|element| ... } -> new_array\narray.map -> new_enumerator"]
+Source code 3.2:ruby-3.2.0/array.c:3825
+
+Calls the block, if given, with each element of self; returns a new Array whose elements are the return values from the block:
+
+a = [:foo, 'bar', 2]
+a1 = a.map {|element| element.class }
+a1 # => [Symbol, String, Integer]
+
+Returns a new Enumerator if no block given:
+a = [:foo, 'bar', 2]
+a1 = a.map
+a1 # => #
+
+Array#collect is an alias for Array#map.
+Examples static VALUE
+rb_ary_collect(VALUE ary)
+{
+long i;
+VALUE collect;
+
+RETURN_SIZED_ENUMERATOR(ary, 0, 0, ary_enum_length);
+collect = rb_ary_new2(RARRAY_LEN(ary));
+for (i = 0; i < RARRAY_LEN(ary); i++) {
+  rb_ary_push(collect, rb_yield(RARRAY_AREF(ary, i)));
+}
+return collect;
+}
+```
+
+2. Chunk texts into meaningful blocks.
+3. Create embeddings for texts.
+4. Store embeddings in a vector database.
+
+Aoororachain uses the Chroma vector database to store and query embeddings.
+
+Here is an example for loading and creating the embeddings. 
+
+```ruby
+require "aoororachain"
+
+# Setup logger.
+Aoororachain.logger = Logger.new($stdout)
+Aoororachain.log_level = Aoororachain::LEVEL_DEBUG
+
+chroma_host = "http://localhost:8000"
+collection_name = "ruby-documentation"
+
+# You can define a custom Parser to clean data and maybe extract metadata.
+# Here is the code of RubyDocParser that does exactly that.
+class RubyDocParser
+def self.parse(text)
+name_match = text.match(/Name (\w+)/)
+constant_match = text.match(/Constant (\w+)/)
+
+object_match = text.match(/Object (\w+)/)
+method_match = text.match(/Method ([\w\[\]\+\=\-\*\%\/]+)/)
+
+metadata = {}
+metadata[:name] = name_match[1] if name_match
+metadata[:constant] = constant_match[1] if constant_match
+metadata[:object] = object_match[1] if object_match
+metadata[:method] = method_match[1] if method_match
+metadata[:lang] = :ruby
+metadata[:version] = "3.2"
+
+text.gsub!(/\s+/, " ").strip!
+[text, metadata]
+end
+end
+
+# A DirectoryLoader points to a path and sets the glob for the files you want to load. 
+# A loader is also specified. FileLoader just opens and reads the file content. 
+# The RubyDocParser is set as well. This is optional in case you data is very nice and needs no pre-processing.
+directory_loader = Aoororachain::Loaders::DirectoryLoader.new(path: "./ruby-docs", glob: "**/*.txt", loader: Aoororachain::Loaders::FileLoader, parser: RubyDocParser)
+files = directory_loader.load
+
+# With your data clean and ready, now it is time to chunk it. The chunk size depends of the context size of the LLMs that you want to use.
+# 512 is a good number to start, don't go lower than that. An overlap can also be specified.
+text_splitter = Aoororachain::RecursiveTextSplitter.new(size: 512, overlap: 0)
+
+texts = []
+files.each do |file|
+texts.concat(text_splitter.split_documents(file))
+end
+
+# The final step is to create and store the embeddings.
+# First, select an embedding model
+model = Aoororachain::Embeddings::LocalPythonEmbedding::MODEL_INSTRUCTOR_L
+# Create an instance of the embedder. device is optional. Possible options are:
+# - cuda. If you have an external GPU
+# - mps. If you have an Apple Sillicon chip (M1 to M2).
+# - cpu or empty. It will use the CPU by default.
+embedder = Aoororachain::Embeddings::LocalPythonEmbedding.new(model:, device: "mps")
+# Configure your Vector database.
+vector_database = Aoororachain::VectorStores::Chroma.new(embedder: embedder, options: {host: chroma_host})
+
+# Embbed your files. This can take a few minutes up to hours, depending on the size of your documents and the model used.
+vector_database.from_documents(texts, index: collection_name)
+```
+
+With embedding loaded in the database, you can use a tool like Chroma UI -**not yet released** - to query documents.
+![chroma-ui](https://github.com/mariochavez/aoororachain/assets/59967/d65dea13-c6ef-452a-9774-8cf3b47c048f)
+
+But it is more useful to query with Aoororachain.
+
+```ruby
+# Define a retriever for the Vector database.
+retriever = Aoororachain::VectorStores::Retriever.new(vector_database)
+
+# Query documents, results by default is 3.
+documents = retriever.search("how can I use the Data class?", results: 4)
+
+# Print retrieved documents and their similarity distance from the question.
+puts documents.map(&:document).join(" ")
+puts documents.map(&:distance)
+```
+
+### Query LLM with context.
+
+With embeddings ready, it is time to create a _chain_ to perform QA Retrieval using the embedded documents as context.
+
+```ruby
+require "aoororachain"
+
+# Setup logger.
+Aoororachain.logger = Logger.new($stdout)
+Aoororachain.log_level = Aoororachain::LEVEL_DEBUG
 
-## Development
+llm_host = "http://localhost:9292"
+chroma_host = "http://localhost:8000"
+collection_name = "ruby-documentation"
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+model = Aoororachain::Embeddings::LocalPythonEmbedding::MODEL_INSTRUCTOR_L
+embedder = Aoororachain::Embeddings::LocalPythonEmbedding.new(model:, device: "mps")
+vector_database = Aoororachain::VectorStores::Chroma.new(embedder: embedder, options: {host: chroma_host, log_level: Chroma::LEVEL_DEBUG})
+vector_database.from_index(collection_name)
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+retriever = Aoororachain::VectorStores::Retriever.new(vector_database)
+
+# Configure the LLM Server
+llm = Aoororachain::Llms::LlamaServer.new(llm_host)
+
+# Create the chain to connect the Vector database retriever with the LLM.
+chain = Aoororachain::Chains::RetrievalQA.new(llm, retriever)
+
+# Create a template for the LLM. Aoororachain does not include any templates because these are model specific. The following template is for the Vicuna model.
+template = "A conversation between a human and an AI assistant. The assistant responds to a question using the context. Context: ===%{context}===. Question: %{prompt}"
+
+response = chain.complete(prompt: "how can I use the Data class to define a new class?", prompt_template: template)
+```
 
 ## Contributing
 

data/lib/aoororachain/chains/retrieval_qa.rb

@@ -9,15 +9,12 @@ module Aoororachain
         @type = type
       end
 
-      def complete(prompt:)
+      def complete(prompt:, prompt_template:)
         context = @retriever.search(prompt)
 
-        system_prompt = "Una conversación entre un humano y un asistente de inteligencia artificial. El asistente response usando el contexto la pregunta. Si no sabes la respuesta, simplemente di que no sabes, no trates de inventar una."
-        context_prompt = "Contexto: #{context.map(&:document).join(" ").tr("\n", " ")}"
-        question_prompt = "Pregunta: #{prompt}"
+        stuff_prompt = prompt_template % {context: context.map(&:document).join(" ").tr("\n", " "), prompt:}
 
-        stuff_prompt = [system_prompt, context_prompt, question_prompt]
-        success, response = @llm.complete(prompt: stuff_prompt.join(". "))
+        success, response = @llm.complete(prompt: stuff_prompt)
 
         if success
           completion = {

data/lib/aoororachain/embeddings/local_python_embedding.rb

@@ -12,9 +12,7 @@ module Aoororachain
         @device = options.delete(:device) || "cpu"
 
         Aoororachain::Util.log_info("Using", data: {model: @model, device: @device})
-        Aoororachain::Util.log_info("This embedding calls Python code using system call. First time initialization might take long due to Python dependencies installation.")
-
-        install_python_dependencies
+        Aoororachain::Util.log_info("This embedding calls Python code using system call.")
       end
 
       def embed_documents(documents, include_metadata: false)
@@ -151,18 +149,6 @@ module Aoororachain
 
         file_path
       end
-
-      def install_python_dependencies
-        stdout_data, stderr_data, exit_code = run_system("pip -q install langchain sentence_transformers InstructorEmbedding")
-
-        if exit_code != 0
-          Aoororachain.log_error("Failed to install Python dependencies: #{stderr_data}")
-          return false
-        end
-
-        Aoororachain::Util.log_debug("Python installed dependencies: #{stdout_data}")
-        true
-      end
     end
   end
 end

data/lib/aoororachain/llms/llama_server.rb

@@ -14,7 +14,7 @@ module Aoororachain
       def complete(prompt:)
         result = LlmClient.completion(prompt)
 
-        [result.success?, result.success? ? result.success.body["response"].gsub(/Usuario:.*Asistente:/, "") : result.failure.message]
+        [result.success?, result.success? ? result.success.body["response"].gsub(/Usuario:.*Asistente:/, "") : result.failure.body]
       end
     end
   end

data/lib/aoororachain/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Aoororachain
-  VERSION = "0.1.2"
+  VERSION = "0.1.4"
 end

data/lib/aoororachain.rb

@@ -4,6 +4,7 @@ require "uri"
 require "json"
 require "logger"
 require "forwardable"
+require "open3"
 require "tempfile"
 
 require_relative "aoororachain/version"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: aoororachain
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.1.4
 platform: ruby
 authors:
 - Mario Alberto Chávez
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2023-06-25 00:00:00.000000000 Z
+date: 2023-07-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: chroma-db
@@ -106,7 +106,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.14
+rubygems_version: 3.4.15
 signing_key:
 specification_version: 4
 summary: Aoororachain for working with LLMs