boxcars 0.1.3 → 0.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1d5ae0de1aba4038723912365d6ac1d50da0e34a40aad508130ecc86e71f91de
-  data.tar.gz: e0430621516358a59b7eeaadfa44fe78eed0b3ef92d929c264435d0946dbce5e
+  metadata.gz: 0fc42578d07a962cd91fbab547ba657842c131729070e57bcf7789aa1648aaee
+  data.tar.gz: 3e3009663c45d58ee616d2d705c74f939b12922b24bceff74d231f5aeaff573d
 SHA512:
-  metadata.gz: a054ad6b3b3caef1c6bb39fbd9a667779c4dc638800e905b717285b8f5dde4a578e3325373176b099af9f4f7d07b6f7803f87fe67c4f771bfce8894ef209188e
-  data.tar.gz: f3a3020bd0c393c25cd430b8702e26a7a39b87127372787c416042da63c6fb90826773ca81a456ba67a1add908790620757d8b9aab4db32269e3391d576c5ea5
+  metadata.gz: d5a2664fb2b4973b57e41f682a9b194394bc196ada00b9c8cf9106a7876b92065cd9f40ff270650219c282143f812241e16fc450a24f61567acaaf413f44150f
+  data.tar.gz: 1d73a0c1519a3834852ffe7108e43e03460cc6e1b99dbcc9c16bf73c7824016d71628878b32394a33b548a0c84de7f9a41feefa1dc922628957eb0251269cd7a

data/CHANGELOG.md

@@ -1,12 +1,14 @@
 # Changelog
 
-## [Unreleased](https://github.com/BoxcarsAI/boxcars/tree/HEAD)
+## [v0.1.3](https://github.com/BoxcarsAI/boxcars/tree/v0.1.3) (2023-02-17)
 
-[Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.1.2...HEAD)
+[Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.1.2...v0.1.3)
 
 **Closed issues:**
 
+- generate changelog automatically [\#12](https://github.com/BoxcarsAI/boxcars/issues/12)
 - Make sure the yard docs are up to date and have coverage [\#7](https://github.com/BoxcarsAI/boxcars/issues/7)
+- Name changes and code movement. [\#6](https://github.com/BoxcarsAI/boxcars/issues/6)
 - Specs need environment variables to be set to run green [\#4](https://github.com/BoxcarsAI/boxcars/issues/4)
 
 **Merged pull requests:**

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    boxcars (0.1.3)
+    boxcars (0.1.4)
       google_search_results (~> 2.2)
       ruby-openai (~> 3.0)
 

data/README.md

@@ -1,15 +1,26 @@
-# Boxcars
+<h2 align="center">Boxcars</h2>
 
-This gem lets you compose new systems with composability using systems like OpenAI, Search, and SQL (and many more).
+<h4 align="center">
+  <a href="https://www.boxcars.ai">Website</a> |
+  <a href="https://www.boxcars.ai/roadmap">Roadmap</a> |
+  <a href="https://www.boxcars.ai/blog">Blog</a> |
+  <a href="https://www.boxcars.ai/en/introduction/">Documentation</a> 
+</h4>
 
-The popular python library langchain is a precursor to this work, but we wanted to put a ruby spin on things and make it easier to get started.
+<p align="center">
+  <a href="https://github.com/BoxcarsAI/boxcars/blob/main/LICENSE.txt"><img src="https://img.shields.io/badge/license-MIT-informational" alt="License"></a>
+</p>
+
+Boxcars is a gem that enables you to create new systems with AI composability, using various concepts such as OpenAI, Search, SQL, Rails Active Record and more (including your concepts).
+
+This work was inspired by the popular Python library Langchain. However, we wanted to give it a Ruby spin and make it more user-friendly for beginners to get started.
 
 ## Concepts
 All of these concepts are in a module named Boxcars:
-- Prompt: a prompt is used by an Engine to generate text results.
-- Engine: an entity that generates text from a Prompt.
-- Boxcar: an encapsulation that does one thing (search, math, SQL, etc) and a many use an Engine to get their work accomplished.cc
-- Train: given a list of Boxcars and an Engine, an answer if found by breaking the promlem into peices for an indvidual Boxcar to solve. This is all joined back together to yield a final result. There is currently only one implementation - ZeroShot. You can construct it directly, or just use `Boxcars::default_train`
+- Prompt: used by an Engine to generate text results.
+- Engine: an entity that generates text from a Prompt. OpenAI's LLM text generatory is the default Engine if no other is specified.
+- Boxcar: an encapsulation of a concept that performs something of interest (such as search, math, or SQL). A can use an Engine to do its work.
+- Train: Given a list of Boxcars and an optionally an Engine, a Train breaks down a problem into pieces for individual Boxcars to solve. The individual results are then combined until a final answer is found. ZeroShot is the only current implementation of Train, and you can either construct it directly or use `Boxcars::train` when you want to build a Train.
 
 ## Installation
 
@@ -29,7 +40,7 @@ Or install it yourself as:
 
 ## Usage
 
-We will document many more examples soon, but here are a couple. The first step is to setup your environment variables for OpenAI and Serp (OPENAI_ACCESS_TOKEN, SERPAPI_API_KEY). If you don't want to set them in your environment, they can be passed in as well to the API.
+We will be adding more examples soon, but here are a couple to get you started. First, you'll need to set up your environment variables for OpenAI and Google SERP (OPENAI_ACCESS_TOKEN, SERPAPI_API_KEY). If you prefer not to set these variables in your environment, you can pass them directly into the API.
 
 In the examples below, we added one rubygem to load the environment at the first line, but depending on what you want, you might not need this.
 ```ruby
@@ -37,6 +48,11 @@ require "dotenv/load"
 require "boxcars"
 ```
 
+Note: if you want to try out the examples below, run this command and then paste in the code segments of interest:
+```bash
+irb -r dotenv/load -r boxcars
+```
+
 ### Direct Boxcar Use
 
 ```ruby
@@ -45,7 +61,7 @@ engine = Boxcars::Openai.new(max_tokens: 256)
 calc = Boxcars::Calculator.new(engine: engine)
 puts calc.run "what is pi to the forth power divided by 22.1?"
 ```
-This segment above (try it by pasting into an `irb` session)
+Produces:
 ```text
 > Entering Calculator#run
 what is pi to the forth power divided by 22.1?
@@ -74,10 +90,10 @@ Here is what we have so far, but please put up a PR with your new ideas.
 ```ruby
 # run a Train for a calculator, and search using default Engine
 boxcars = [Boxcars::Calculator.new, Boxcars::Serp.new]
-train = Boxcars.default_train.new(boxcars: boxcars)
+train = Boxcars.train.new(boxcars: boxcars)
 puts train.run "What is pi times the square root of the average temperature in Austin TX in January?"
 ```
-This outputs:
+Produces:
 ```text
 > Entering Zero Shot#run
 What is pi times the square root of the average temperature in Austin TX in January?

data/boxcars.gemspec

@@ -23,7 +23,7 @@ Gem::Specification.new do |spec|
   # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
   spec.files = Dir.chdir(File.expand_path(__dir__)) do
     `git ls-files -z`.split("\x0").reject do |f|
-      (f == __FILE__) || f.match(%r{\A(?:(?:test|spec|features)/|\.(?:git|travis|circleci)|appveyor)})
+      (f == __FILE__) || f.match(%r{\A(?:(?:test|spec|features|notebooks)/|\.(?:git|travis|circleci)|appveyor)})
     end
   end
   spec.bindir = "exe"

data/lib/boxcars/boxcar/active_record.rb

@@ -0,0 +1,130 @@
+# frozen_string_literal: true
+
+# Boxcars is a framework for running a series of tools to get an answer to a question.
+module Boxcars
+  # A Boxcar that interprets a prompt and executes SQL code to get answers
+  class ActiveRecord < EngineBoxcar
+    # the description of this engine boxcar
+    ARDESC = "useful for when you need to query a database for an application named %<name>s."
+    LOCKED_OUT_MODELS = %w[ActiveRecord::SchemaMigration ActiveRecord::InternalMetadata ApplicationRecord].freeze
+    attr_accessor :connection, :input_key, :requested_models
+    attr_reader :except_models
+
+    # @param engine [Boxcars::Engine] The engine to user for this boxcar. Can be inherited from a train if nil.
+    # @param models [Array<ActiveRecord::Model>] The models to use for this boxcar. Will use all if nil.
+    # @param input_key [Symbol] The key to use for the input. Defaults to :question.
+    # @param output_key [Symbol] The key to use for the output. Defaults to :answer.
+    # @param kwargs [Hash] Any other keyword arguments to pass to the parent class. This can include
+    #   :name, :description and :prompt
+    def initialize(engine: nil, models: nil, input_key: :question, output_key: :answer, **kwargs)
+      if models.is_a?(Array) && models.length.positive?
+        @requested_models = models
+        models.each do |m|
+          raise ArgumentError, "model #{m} needs to be an Active Record model" unless m.ancestors.include?(::ActiveRecord::Base)
+        end
+      elsif models
+        raise ArgumentError, "models needs to be an array of Active Record models"
+      end
+      @except_models = LOCKED_OUT_MODELS + kwargs[:except_models].to_a
+      @input_key = input_key
+      the_prompt = kwargs[prompt] || my_prompt
+      name = kwargs[:name] || "Data"
+      super(name: name,
+            description: kwargs[:description] || format(ARDESC, name: name),
+            engine: engine,
+            prompt: the_prompt,
+            output_key: output_key)
+    end
+
+    # the input keys for the prompt
+    # @return [Array<Symbol>] The input keys for the prompt.
+    def input_keys
+      [input_key]
+    end
+
+    # the output keys for the prompt
+    # @return [Array<Symbol>] The output keys for the prompt.
+    def output_keys
+      [output_key]
+    end
+
+    # call the boxcar
+    # @param inputs [Hash] The inputs to the boxcar.
+    # @return [Hash] The outputs from the boxcar.
+    def call(inputs:)
+      t = predict(question: inputs[input_key], top_k: 5, model_info: model_info, stop: ["Answer:"]).strip
+      answer = get_answer(t)
+      puts answer.colorize(:magenta)
+      { output_key => answer }
+    end
+
+    private
+
+    def wanted_models
+      the_models = requested_models || ::ActiveRecord::Base.descendants
+      the_models.reject { |m| except_models.include?(m.name) }
+    end
+
+    def models
+      models = wanted_models.map(&:name)
+      models.join(", ")
+    end
+
+    def model_info
+      models = wanted_models
+      models.pretty_inspect
+    end
+
+    def get_active_record_answer(text)
+      code = text[/^ARCode: (.*)/, 1]
+      puts code.colorize(:yellow)
+      begin
+        # rubocop:disable Security/Eval
+        output = eval code
+        # rubocop:enable Security/Eval
+        output = output.first if output.is_a?(Array) && output.length == 1
+        "Answer: #{output.inspect}"
+      rescue StandardError => e
+        "Error: #{e.message}"
+      end
+    end
+
+    def get_answer(text)
+      case text
+      when /^ARCode:/
+        get_active_record_answer(text)
+      when /^Answer:/
+        text
+      else
+        raise Boxcars::Error "Unknown format from engine: #{text}"
+      end
+    end
+
+    TEMPLATE = <<~IPT
+      Given an input question, first create a syntactically correct Rails Active Record code to run,
+      then look at the results of the code and return the answer. Unless the user specifies
+      in her question a specific number of examples she wishes to obtain, limit your code
+      to at most %<top_k>s results.
+
+      Never query for all the columns from a specific model, only ask for a the few relevant attributes given the question.
+
+      Pay attention to use only the attribute names that you can see in the model description. Be careful to not query for attributes that do not exist.
+      Also, pay attention to which attribute is in which model.
+
+      Use the following format:
+      Question: "Question here"
+      ARCode: "Active Record code to run"
+      Answer: "Final answer here"
+
+      Only use the following Active Record models:
+      %<model_info>s
+
+      Question: %<question>s
+    IPT
+
+    # The prompt to use for the engine.
+    def my_prompt
+      @my_prompt ||= Prompt.new(input_variables: [:question, :top_k, :model_info], template: TEMPLATE)
+    end
+  end
+end

data/lib/boxcars/boxcar/engine_boxcar.rb

@@ -13,7 +13,7 @@ module Boxcars
     # @param engine [Boxcars::Engine] The engine to user for this boxcar. Can be inherited from a train if nil.
     def initialize(prompt:, engine: nil, output_key: "text", name: nil, description: nil)
       @prompt = prompt
-      @engine = engine || Boxcars.default_engine.new
+      @engine = engine || Boxcars.engine.new
       @output_key = output_key
       super(name: name, description: description)
     end

data/lib/boxcars/boxcar/{serp.rb → google_search.rb}

@@ -2,8 +2,9 @@
 
 require 'google_search_results'
 module Boxcars
-  # A Boxcar that uses the Google SerpAPI to get answers to questions.
-  class Serp < Boxcar
+  # A Boxcar that uses the Google SERP API to get answers to questions.
+  # It looks through SERP (search engine results page) results to find the answer.
+  class GoogleSearch < Boxcar
     # the description of this boxcar
     SERPDESC = "useful for when you need to answer questions about current events." \
                "You should ask targeted questions"
@@ -15,14 +16,14 @@ module Boxcars
     def initialize(name: "Search", description: SERPDESC, serpapi_api_key: "not set")
       super(name: name, description: description)
       api_key = Boxcars.configuration.serpapi_api_key(serpapi_api_key: serpapi_api_key)
-      GoogleSearch.api_key = api_key
+      ::GoogleSearch.api_key = api_key
     end
 
     # Get an answer from Google using the SerpAPI.
     # @param question [String] The question to ask Google.
     # @return [String] The answer to the question.
     def run(question)
-      search = GoogleSearch.new(q: question)
+      search = ::GoogleSearch.new(q: question)
       rv = find_answer(search.get_hash)
       puts "Question: #{question}"
       puts "Answer: #{rv}"
@@ -33,7 +34,7 @@ module Boxcars
     # @param question [String] The question to ask Google.
     # @return [String] The location found.
     def get_location(question)
-      search = GoogleSearch.new(q: question, limit: 3)
+      search = ::GoogleSearch.new(q: question, limit: 3)
       rv = search.get_location
       puts "Question: #{question}"
       puts "Answer: #{rv}"

data/lib/boxcars/boxcar/sql.rb

@@ -5,7 +5,7 @@ module Boxcars
   # A Boxcar that interprets a prompt and executes SQL code to get answers
   class SQL < EngineBoxcar
     # the description of this engine boxcar
-    SQLDESC = "useful for when you need to query a SQL database"
+    SQLDESC = "useful for when you need to query a database for %<name>s."
     attr_accessor :connection, :input_key
 
     # @param connection [ActiveRecord::Connection] The SQL connection to use for this boxcar.
@@ -14,12 +14,13 @@ module Boxcars
     # @param output_key [Symbol] The key to use for the output. Defaults to :answer.
     # @param kwargs [Hash] Any other keyword arguments to pass to the parent class. This can include
     #   :name, :description and :prompt
-    def initialize(connection:, engine: nil, input_key: :question, output_key: :answer, **kwargs)
-      @connection = connection
+    def initialize(connection: nil, engine: nil, input_key: :question, output_key: :answer, **kwargs)
+      @connection = connection || ::ActiveRecord::Base.connection
       @input_key = input_key
       the_prompt = kwargs[prompt] || my_prompt
-      super(name: kwargs[:name] || "SQLdatabase",
-            description: kwargs[:description] || SQLDESC,
+      name = kwargs[:name] || "data"
+      super(name: name,
+            description: kwargs[:description] || format(SQLDESC, name: name),
             engine: engine,
             prompt: the_prompt,
             output_key: output_key)
@@ -73,7 +74,6 @@ module Boxcars
       code = text[/^SQLQuery: (.*)/, 1]
       puts code.colorize(:yellow)
       output = connection.exec_query(code).to_a
-      puts "Answer: #{output}"
       "Answer: #{output}"
     end
 
@@ -116,18 +116,5 @@ module Boxcars
     def my_prompt
       @my_prompt ||= Prompt.new(input_variables: [:question, :dialect, :top_k], template: TEMPLATE)
     end
-
-    # DECIDER_TEMPLATE = <<~DPT
-    #   Given the below input question and list of potential tables, output a comma separated list of the table names that may
-    #   be necessary to answer this question.
-    #   Question: %<query>s
-    #   Table Names: %<table_names>s
-    #   Relevant Table Names:
-    # DPT
-    # DECIDER_PROMPT = Prompt.new(
-    #   input_variables: %i[query table_names],
-    #   template: DECIDER_TEMPLATE,
-    #   output_parser: CommaSeparatedListOutputParser
-    # )
   end
 end

data/lib/boxcars/boxcar.rb

@@ -116,5 +116,6 @@ end
 
 require "boxcars/boxcar/engine_boxcar"
 require "boxcars/boxcar/calculator"
-require "boxcars/boxcar/serp"
+require "boxcars/boxcar/google_search"
 require "boxcars/boxcar/sql"
+require "boxcars/boxcar/active_record"

data/lib/boxcars/train.rb

@@ -205,7 +205,7 @@ module Boxcars
           observation = "#{output.boxcar} is not a valid boxcar, try another one."
           return_direct = false
         end
-        puts "#Observation: #{observation}".colorize(:green)
+        puts "Observation: #{observation}".colorize(:green)
         intermediate_steps.append([output, observation])
         if return_direct
           output = TrainFinish.new({ return_values[0] => observation }, "")

data/lib/boxcars/version.rb

@@ -2,5 +2,5 @@
 
 module Boxcars
   # The current version of the gem.
-  VERSION = "0.1.3"
+  VERSION = "0.1.4"
 end

data/lib/boxcars.rb

@@ -36,7 +36,7 @@ module Boxcars
   # Configuration contains gem settings
   class Configuration
     attr_writer :openai_access_token, :serpapi_api_key
-    attr_accessor :organization_id, :logger, :log_prompts
+    attr_accessor :organization_id, :logger, :log_prompts, :default_train, :default_engine
 
     def initialize
       @organization_id = nil
@@ -96,13 +96,13 @@ module Boxcars
   end
 
   # Return the default Train class.
-  def self.default_train
-    Boxcars::ZeroShot
+  def self.train
+    configuration.default_train || Boxcars::ZeroShot
   end
 
   # Return the default Engine class.
-  def self.default_engine
-    Boxcars::Openai
+  def self.engine
+    configuration.default_engine || Boxcars::Openai
   end
 end
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: boxcars
 version: !ruby/object:Gem::Version
-  version: 0.1.3
+  version: 0.1.4
 platform: ruby
 authors:
 - Francis Sullivan
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2023-02-17 00:00:00.000000000 Z
+date: 2023-02-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: debug
@@ -102,9 +102,10 @@ files:
 - boxcars.gemspec
 - lib/boxcars.rb
 - lib/boxcars/boxcar.rb
+- lib/boxcars/boxcar/active_record.rb
 - lib/boxcars/boxcar/calculator.rb
 - lib/boxcars/boxcar/engine_boxcar.rb
-- lib/boxcars/boxcar/serp.rb
+- lib/boxcars/boxcar/google_search.rb
 - lib/boxcars/boxcar/sql.rb
 - lib/boxcars/engine.rb
 - lib/boxcars/engine/engine_result.rb