boxcars 0.1.6 → 0.1.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 666f9b91451e8b6db61eb1162cb6508a566a46b676d646f1fb0000175714cede
-  data.tar.gz: bee7b55be2288bf9361012bd562f88d91f64ce7972b0cf5c495a5fce87eea77c
+  metadata.gz: 3de375ad403bb62fb6af1045aff6a6115cbb598c3747fe48f882fb40483289e0
+  data.tar.gz: 343a3077d72ce21a56d8045701a8a3bd681c5fe91fd8ffbca2cc14270b92f3bb
 SHA512:
-  metadata.gz: '0681f5bd00bbfe2afd67011d76c561640a0575a9656b20913d3c60193208ea33361003ee652d01802b49702b106bd4f3d5bc91c1c7f2023bc2e8224edca14a8d'
-  data.tar.gz: 794fe20c2f9b8f14402db177bdee3d3e5318ab652e69c02929f9ae79d6daca096ae386a4e85730da4004927ab31aff53cc3a8d960ca661d6a5dd192432ffeabb
+  metadata.gz: efc72210fb52599b7c6278502a5b4143d995e671c62c7f6c090cbddf2a8926b231b0d7ab0aa28fa1325324ded1ebac5cac4e10e608ae59077a431a20f9d62f23
+  data.tar.gz: eeb00a993d7bbf64ac5c7461a1e31f241407c58a19a919c0853ade7e4a3bf717ab4f69331c5bc5d8c0049a89c53353d311d41616352131379b7e87b8ec3e491d

data/.env_sample

@@ -0,0 +1,2 @@
+openai_access_token: ''
+serpapi_api_key: ''

data/CHANGELOG.md

@@ -1,8 +1,20 @@
 # Changelog
 
-## [Unreleased](https://github.com/BoxcarsAI/boxcars/tree/HEAD)
+## [v0.1.7](https://github.com/BoxcarsAI/boxcars/tree/v0.1.7) (2023-02-27)
 
-[Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.1.5...HEAD)
+[Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.1.6...v0.1.7)
+
+**Implemented enhancements:**
+
+- figure out logging [\#10](https://github.com/BoxcarsAI/boxcars/issues/10)
+
+**Merged pull requests:**
+
+- Fix typos in README concepts [\#26](https://github.com/BoxcarsAI/boxcars/pull/26) ([MasterOdin](https://github.com/MasterOdin))
+
+## [v0.1.6](https://github.com/BoxcarsAI/boxcars/tree/v0.1.6) (2023-02-24)
+
+[Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.1.5...v0.1.6)
 
 **Implemented enhancements:**
 

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    boxcars (0.1.6)
+    boxcars (0.1.8)
       google_search_results (~> 2.2)
       ruby-openai (~> 3.0)
 

data/README.md

@@ -17,10 +17,11 @@ This gem was inspired by the popular Python library Langchain. However, we wante
 
 ## Concepts
 All of these concepts are in a module named Boxcars:
-- Prompt: used by an Engine to generate text results.
-- Engine: an entity that generates text from a Prompt. OpenAI's LLM text generatory is the default Engine if no other is specified.
-- Boxcar: an encapsulation of a concept that performs something of interest (such as search, math, or SQL). A can use an Engine to do its work.
-- Train: Given a list of Boxcars and an optionally an Engine, a Train breaks down a problem into pieces for individual Boxcars to solve. The individual results are then combined until a final answer is found. ZeroShot is the only current implementation of Train, and you can either construct it directly or use `Boxcars::train` when you want to build a Train.
+
+- Boxcar - an encapsulation that performs something of interest (such as search, math, SQL or an Active Record Query). A Boxcar can use an Engine (described below) to do its work.
+- Train - Given a list of Boxcars and optionally an Engine, a Train breaks down a problem into pieces for individual Boxcars to solve. The individual results are then combined until a final answer is found. ZeroShot is the only current implementation of Train (but we are adding more soon), and you can either construct it directly or use `Boxcars::train` when you want to build a Train.
+- Prompt - used by an Engine to generate text results. Most of the Boxcars have built-in prompts, so you only need to worry about these if you are extending the system.
+- Engine - an entity that generates text from a Prompt. OpenAI's LLM text generator is the default Engine if no other is specified.
 
 ## Installation
 
@@ -86,6 +87,7 @@ Here is what we have so far, but please put up a PR with your new ideas.
 - GoogleSearch: uses the SERP API to do seaches
 - Calculator: uses an Engine to generate ruby code to do math
 - SQL: given an ActiveRecord connection, it will generate and run sql statments from a prompt.
+- ActiveRecord: given an ActiveRecord connection, it will generate and run ActiveRecord statements from a prompt.
 
 ### Run a list of Boxcars
 ```ruby
@@ -114,7 +116,18 @@ Final Answer: 201.06
 < Exiting Zero Shot#run
 201.06
 ```
+### More Examples
+See [this](https://github.com/BoxcarsAI/boxcars/blob/main/notebooks/boxcars_examples.ipynb) Jupyter Notebook for more examples.
+
+Note, some folks that we talked to didn't know that you could run Ruby Jupyter notebooks. [You can](https://github.com/SciRuby/iruby).
+
+### Logging
+If you use this in a Rails application, or configure `Boxcars.configuration.logger = your_logger`, logging will go to your log file.
+
+Also, if you set this flag: `Boxcars.configuration.lop_prompts = true`
+The actual prompts handed to the connected Engine will be logged. This is off by default because it is very wordy, but handy if you are debugging prompts.
 
+Otherwise, we print to standard out.
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
@@ -123,7 +136,7 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/boxcars. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/boxcars/blob/main/CODE_OF_CONDUCT.md).
+Bug reports and pull requests are welcome on GitHub at https://github.com/BoxcarsAI/boxcars. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/BoxcarsAI/boxcars/blob/main/CODE_OF_CONDUCT.md).
 
 ## License
 
@@ -131,4 +144,4 @@ The gem is available as open source under the terms of the [MIT License](https:/
 
 ## Code of Conduct
 
-Everyone interacting in the Boxcars project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/boxcars/blob/main/CODE_OF_CONDUCT.md).
+Everyone interacting in the Boxcars project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/BoxcarsAI/boxcars/blob/main/CODE_OF_CONDUCT.md).

data/lib/boxcars/boxcar/active_record.rb

@@ -7,7 +7,7 @@ module Boxcars
     # the description of this engine boxcar
     ARDESC = "useful for when you need to query a database for an application named %<name>s."
     LOCKED_OUT_MODELS = %w[ActiveRecord::SchemaMigration ActiveRecord::InternalMetadata ApplicationRecord].freeze
-    attr_accessor :connection, :input_key, :requested_models, :read_only, :approval_callback
+    attr_accessor :connection, :requested_models, :read_only, :approval_callback
     attr_reader :except_models
 
     # @param engine [Boxcars::Engine] The engine to user for this boxcar. Can be inherited from a train if nil.
@@ -15,43 +15,25 @@ module Boxcars
     # @param read_only [Boolean] Whether to use read only models. Defaults to true unless you pass an approval function.
     # @param approval_callback [Proc] A function to call to approve changes. Defaults to nil.
     # @param kwargs [Hash] Any other keyword arguments. These can include:
-    #   :name, :description, :prompt, :input_key, :output_key and :except_models
+    #   :name, :description, :prompt, :except_models, :top_k, and :stop
     def initialize(engine: nil, models: nil, read_only: nil, approval_callback: nil, **kwargs)
       check_models(models)
       @except_models = LOCKED_OUT_MODELS + kwargs[:except_models].to_a
       @approval_callback = approval_callback
       @read_only = read_only.nil? ? !approval_callback : read_only
-      @input_key = kwargs[:input_key] || :question
-      @output_key = kwargs[:output_key] || :answer
       the_prompt = kwargs[prompt] || my_prompt
       name = kwargs[:name] || "Data"
+      kwargs[:stop] ||= ["Answer:"]
       super(name: name,
             description: kwargs[:description] || format(ARDESC, name: name),
             engine: engine,
             prompt: the_prompt,
-            output_key: output_key)
+            **kwargs)
     end
 
-    # the input keys for the prompt
-    # @return [Array<Symbol>] The input keys for the prompt.
-    def input_keys
-      [input_key]
-    end
-
-    # the output keys for the prompt
-    # @return [Array<Symbol>] The output keys for the prompt.
-    def output_keys
-      [output_key]
-    end
-
-    # call the boxcar
-    # @param inputs [Hash] The inputs to the boxcar.
-    # @return [Hash] The outputs from the boxcar.
-    def call(inputs:)
-      t = predict(question: inputs[input_key], top_k: 5, model_info: model_info, stop: ["Answer:"]).strip
-      answer = get_answer(t)
-      puts answer.colorize(:magenta)
-      { output_key => answer }
+    # @return Hash The additional variables for this boxcar.
+    def prediction_additional
+      { model_info: model_info }.merge super
     end
 
     private
@@ -83,7 +65,7 @@ module Boxcars
 
     def model_info
       models = wanted_models
-      models.pretty_inspect
+      models.inspect
     end
 
     # to be safe, we wrap the code in a transaction and rollback
@@ -108,7 +90,7 @@ module Boxcars
 
       bad_words.each do |w|
         if word_list.include?(w)
-          puts "code included destructive instruction: #{w} #{code}".colorize(:red)
+          Boxcars.info "code included destructive instruction: #{w} #{code}", :red
           return false
         end
       end
@@ -128,7 +110,7 @@ module Boxcars
       return 0 unless changes_code
 
       rollback_after_running do
-        puts "computing change count with: #{changes_code}".colorize(:yellow)
+        Boxcars.debug "computing change count with: #{changes_code}", :yellow
         evaluate_input changes_code
       end
     end
@@ -138,7 +120,7 @@ module Boxcars
       changes = change_count(changes_code)
       return true unless changes&.positive?
 
-      puts "Pending Changes: #{changes}".colorize(:yellow, style: :bold)
+      Boxcars.debug "Pending Changes: #{changes}", :yellow, style: :bold
       change_str = "#{changes} change#{'s' if changes.to_i > 1}"
       raise SecurityError, "Can not run code that makes #{change_str} in read-only mode" if read_only?
 
@@ -148,7 +130,7 @@ module Boxcars
     end
 
     def run_active_record_code(code)
-      puts code.colorize(:yellow)
+      Boxcars.debug code, :yellow
       if read_only?
         rollback_after_running do
           evaluate_input code
@@ -167,13 +149,12 @@ module Boxcars
       output = 0 if output.is_a?(Array) && output.empty?
       output = output.first if output.is_a?(Array) && output.length == 1
       output = output[output.keys.first] if output.is_a?(Hash) && output.length == 1
-      "Answer: #{output.inspect}"
+      "Answer: #{output.to_json}"
     rescue StandardError => e
       "Error: #{e.message}"
     end
 
     def get_answer(text)
-      # debugger
       case text
       when /^ARCode:/
         get_active_record_answer(text)
@@ -209,7 +190,8 @@ module Boxcars
 
     # The prompt to use for the engine.
     def my_prompt
-      @my_prompt ||= Prompt.new(input_variables: [:question, :top_k, :model_info], template: TEMPLATE)
+      @my_prompt ||= Prompt.new(input_variables: [:question], other_inputs: [:top_k], output_variables: [:answer],
+                                template: TEMPLATE)
     end
   end
 end

data/lib/boxcars/boxcar/calculator.rb

@@ -6,42 +6,19 @@ module Boxcars
   class Calculator < EngineBoxcar
     # the description of this engine boxcar
     CALCDESC = "useful for when you need to answer questions about math"
-    attr_accessor :input_key
 
     # @param engine [Boxcars::Engine] The engine to user for this boxcar. Can be inherited from a train if nil.
     # @param prompt [Boxcars::Prompt] The prompt to use for this boxcar. Defaults to built-in prompt.
-    # @param input_key [Symbol] The key to use for the input. Defaults to :question.
-    # @param output_key [Symbol] The key to use for the output. Defaults to :answer.
     # @param kwargs [Hash] Any other keyword arguments to pass to the parent class.
-    def initialize(engine: nil, prompt: nil, input_key: :question, output_key: :answer, **kwargs)
-      # def initialize(engine:, prompt: my_prompt, input_key: :question, output_key: :answer, **kwargs)
-      @input_key = input_key
+    def initialize(engine: nil, prompt: nil, **kwargs)
+      # def initialize(engine:, prompt: my_prompt, output_key: :answer, **kwargs)
       the_prompt = prompt || my_prompt
+      kwargs[:stop] ||= ["```output"]
       super(name: kwargs[:name] || "Calculator",
             description: kwargs[:description] || CALCDESC,
             engine: engine,
             prompt: the_prompt,
-            output_key: output_key)
-    end
-
-    # the prompt input keys
-    def input_keys
-      [input_key]
-    end
-
-    # the output keys
-    def output_keys
-      [output_key]
-    end
-
-    # call the calculator
-    # @param inputs [Hash] The inputs to the boxcar.
-    # @return [Hash] The outputs from the boxcar.
-    def call(inputs:)
-      t = predict(question: inputs[input_key], stop: ["```output"]).strip
-      answer = get_answer(t)
-      puts answer.colorize(:magenta)
-      { output_key => answer }
+            **kwargs)
     end
 
     private
@@ -104,7 +81,7 @@ module Boxcars
 
     # The prompt to use for the engine.
     def my_prompt
-      @my_prompt ||= Prompt.new(input_variables: [:question], template: TEMPLATE)
+      @my_prompt ||= Prompt.new(input_variables: [:question], output_variables: [:answer], template: TEMPLATE)
     end
   end
 end

data/lib/boxcars/boxcar/engine_boxcar.rb

@@ -4,17 +4,18 @@
 module Boxcars
   # For Boxcars that use an engine to do their work.
   class EngineBoxcar < Boxcar
-    attr_accessor :prompt, :engine, :output_key
+    attr_accessor :prompt, :engine, :top_k, :stop
 
     # A Boxcar is a container for a single tool to run.
     # @param prompt [Boxcars::Prompt] The prompt to use for this boxcar with sane defaults.
     # @param name [String] The name of the boxcar. Defaults to classname.
     # @param description [String] A description of the boxcar.
     # @param engine [Boxcars::Engine] The engine to user for this boxcar. Can be inherited from a train if nil.
-    def initialize(prompt:, engine: nil, output_key: "text", name: nil, description: nil)
+    def initialize(prompt:, engine: nil, name: nil, description: nil, **kwargs)
       @prompt = prompt
       @engine = engine || Boxcars.engine.new
-      @output_key = output_key
+      @top_k = kwargs[:top_k] || 5
+      @stop = kwargs[:stop] || ["Answer:"]
       super(name: name, description: description)
     end
 
@@ -23,9 +24,14 @@ module Boxcars
       prompt.input_variables
     end
 
+    # the first input key for the prompt
+    def input_key
+      input_keys.first
+    end
+
     # output keys
     def output_keys
-      [output_key]
+      prompt.output_variables
     end
 
     # generate a response from the engine
@@ -35,8 +41,9 @@ module Boxcars
       stop = input_list[0][:stop]
       prompts = []
       input_list.each do |inputs|
+        # prompt.missing_variables?(inputs)
         new_prompt = prompt.format(**inputs)
-        puts "Prompt after formatting:\n#{new_prompt.colorize(:cyan)}" if Boxcars.configuration.log_prompts
+        Boxcars.debug("Prompt after formatting:\n#{new_prompt}", :cyan) if Boxcars.configuration.log_prompts
         prompts.push(new_prompt)
       end
       engine.generate(prompts: prompts, stop: stop)
@@ -48,7 +55,7 @@ module Boxcars
     def apply(input_list:)
       response = generate(input_list: input_list)
       response.generations.to_h do |generation|
-        [output_key, generation[0].text]
+        [output_keys.first, generation[0].text]
       end
     end
 
@@ -56,7 +63,7 @@ module Boxcars
     # @param kwargs [Hash] A hash of input values to use for the prompt.
     # @return [String] The output value.
     def predict(**kwargs)
-      apply(input_list: [kwargs])[output_key]
+      apply(input_list: [kwargs])[output_keys.first]
     end
 
     # predict a response from the engine and parse it
@@ -77,7 +84,7 @@ module Boxcars
     def apply_and_parse(input_list:)
       result = apply(input_list: input_list)
       if prompt.output_parser
-        result.map { |r| prompt.output_parser.parse(r[output_key]) }
+        result.map { |r| prompt.output_parser.parse(r[output_keys.first]) }
       else
         result
       end
@@ -90,5 +97,32 @@ module Boxcars
 
       raise Boxcars::ArgumentError, "run not supported when there is not exactly one output key. Got #{output_keys}."
     end
+
+    # call the boxcar
+    # @param inputs [Hash] The inputs to the boxcar.
+    # @return [Hash] The outputs from the boxcar.
+    def call(inputs:)
+      t = predict(**prediction_variables(inputs)).strip
+      answer = get_answer(t)
+      Boxcars.debug answer, :magenta
+      { output_keys.first => answer }
+    end
+
+    # @param inputs [Hash] The inputs to the boxcar.
+    # @return Hash The input variable for this boxcar.
+    def prediction_input(inputs)
+      { input_key => inputs[input_key] }
+    end
+
+    # @return Hash The additional variables for this boxcar.
+    def prediction_additional
+      { stop: stop, top_k: top_k }
+    end
+
+    # @param inputs [Hash] The inputs to the boxcar.
+    # @return Hash The variables for this boxcar.
+    def prediction_variables(inputs)
+      prediction_input(inputs).merge(prediction_additional)
+    end
   end
 end

data/lib/boxcars/boxcar/sql.rb

@@ -6,46 +6,26 @@ module Boxcars
   class SQL < EngineBoxcar
     # the description of this engine boxcar
     SQLDESC = "useful for when you need to query a database for %<name>s."
-    attr_accessor :connection, :input_key
+    attr_accessor :connection
 
     # @param connection [ActiveRecord::Connection] The SQL connection to use for this boxcar.
     # @param engine [Boxcars::Engine] The engine to user for this boxcar. Can be inherited from a train if nil.
-    # @param input_key [Symbol] The key to use for the input. Defaults to :question.
-    # @param output_key [Symbol] The key to use for the output. Defaults to :answer.
     # @param kwargs [Hash] Any other keyword arguments to pass to the parent class. This can include
-    #   :name, :description and :prompt
-    def initialize(connection: nil, engine: nil, input_key: :question, output_key: :answer, **kwargs)
+    #   :name, :description, :prompt and :top_k
+    def initialize(connection: nil, engine: nil, **kwargs)
       @connection = connection || ::ActiveRecord::Base.connection
-      @input_key = input_key
       the_prompt = kwargs[prompt] || my_prompt
-      name = kwargs[:name] || "data"
+      kwargs[:stop] ||= ["Answer:"]
+      name = kwargs[:name] || "database"
       super(name: name,
             description: kwargs[:description] || format(SQLDESC, name: name),
             engine: engine,
-            prompt: the_prompt,
-            output_key: output_key)
+            prompt: the_prompt)
     end
 
-    # the input keys for the prompt
-    # @return [Array<Symbol>] The input keys for the prompt.
-    def input_keys
-      [input_key]
-    end
-
-    # the output keys for the prompt
-    # @return [Array<Symbol>] The output keys for the prompt.
-    def output_keys
-      [output_key]
-    end
-
-    # call the boxcar
-    # @param inputs [Hash] The inputs to the boxcar.
-    # @return [Hash] The outputs from the boxcar.
-    def call(inputs:)
-      t = predict(question: inputs[input_key], dialect: dialect, top_k: 5, table_info: schema, stop: ["Answer:"]).strip
-      answer = get_answer(t)
-      puts answer.colorize(:magenta)
-      { output_key => answer }
+    # @return Hash The additional variables for this boxcar.
+    def prediction_additional
+      { schema: schema, dialect: dialect }.merge super
     end
 
     private
@@ -66,13 +46,12 @@ module Boxcars
     end
 
     def dialect
-      # connection.instance_variable_get "@config"[:adapter]
       connection.class.name.split("::").last.sub("Adapter", "")
     end
 
     def get_embedded_sql_answer(text)
       code = text[/^SQLQuery: (.*)/, 1]
-      puts code.colorize(:yellow)
+      Boxcars.debug code, :yellow
       output = connection.exec_query(code).to_a
       "Answer: #{output}"
     end
@@ -107,14 +86,18 @@ module Boxcars
       Answer: "Final answer here"
 
       Only use the following tables:
-      %<table_info>s
+      %<schema>s
 
       Question: %<question>s
     IPT
 
     # The prompt to use for the engine.
     def my_prompt
-      @my_prompt ||= Prompt.new(input_variables: [:question, :dialect, :top_k], template: TEMPLATE)
+      @my_prompt ||= Prompt.new(
+        input_variables: [:question],
+        other_inputs: [:top_k, :dialect, :table_info],
+        output_variables: [:answer],
+        template: TEMPLATE)
     end
   end
 end

data/lib/boxcars/boxcar.rb

@@ -62,9 +62,9 @@ module Boxcars
     # you can pass one or the other, but not both.
     # @return [String] The answer to the question.
     def run(*args, **kwargs)
-      puts "> Entering #{name}#run".colorize(:gray, style: :bold)
+      Boxcars.info "> Entering #{name}#run", :gray, style: :bold
       rv = do_run(*args, **kwargs)
-      puts "< Exiting #{name}#run".colorize(:gray, style: :bold)
+      Boxcars.info "< Exiting #{name}#run", :gray, style: :bold
       rv
     end
 
@@ -77,7 +77,7 @@ module Boxcars
       begin
         output = call(inputs: inputs)
       rescue StandardError => e
-        puts "Error in #{name} boxcar#call: #{e}".colorize(:red)
+        Boxcars.error "Error in #{name} boxcar#call: #{e}", :red
         raise e
       end
       validate_outputs(outputs: output.keys)
@@ -101,7 +101,7 @@ module Boxcars
 
     def our_inputs(inputs)
       if inputs.is_a?(String)
-        puts inputs.colorize(:blue) # the question
+        Boxcars.info inputs, :blue # the question
         if input_keys.length != 1
           raise Boxcars::ArgumentError, "A single string input was passed in, but this boxcar expects " \
                                         "multiple inputs (#{input_keys}). When a boxcar expects " \

data/lib/boxcars/engine/openai.rb

@@ -83,6 +83,25 @@ module Boxcars
       end
     end
 
+    # make sure we got a valid response
+    # @param response [Hash] The response to check.
+    # @param must_haves [Array<String>] The keys that must be in the response. Defaults to %w[choices].
+    # @raise [KeyError] if there is an issue with the access token.
+    # @raise [ValueError] if the response is not valid.
+    def check_response(response, must_haves: %w[choices])
+      if response['error']
+        code = response.dig('error', 'code')
+        msg = response.dig('error', 'message') || 'unknown error'
+        raise KeyError, "OPENAI_ACCESS_TOKEN not valid" if code == 'invalid_api_key'
+
+        raise ValueError, "OpenAI error: #{msg}"
+      end
+
+      must_haves.each do |key|
+        raise ValueError, "Expecting key #{key} in response" unless response.key?(key)
+      end
+    end
+
     # Call out to OpenAI's endpoint with k unique prompts.
     # @param prompts [Array<String>] The prompts to pass into the model.
     # @param stop [Array<String>] Optional list of stop words to use when generating.
@@ -98,6 +117,7 @@ module Boxcars
       sub_prompts = prompts.each_slice(batch_size).to_a
       sub_prompts.each do |sprompts|
         response = client(prompt: sprompts, **params)
+        check_response(response)
         choices.concat(response["choices"])
         keys_to_use = inkeys & response["usage"].keys
         keys_to_use.each { |key| token_usage[key] = token_usage[key].to_i + response["usage"][key] }

data/lib/boxcars/prompt.rb

@@ -3,25 +3,44 @@
 module Boxcars
   # used by Boxcars that have engine's to create a prompt.
   class Prompt
-    attr_reader :template, :input_variables, :output_variables
+    attr_reader :template, :input_variables, :other_inputs, :output_variables
 
     # @param template [String] The template to use for the prompt.
-    # @param input_variables [Array<Symbol>] The input vars to use for the prompt.
-    # @param output_variables [Array<Symbol>] The output vars to use for the prompt. Defaults to [:agent_scratchpad]
-    def initialize(template:, input_variables:, output_variables: [:agent_scratchpad])
+    # @param input_variables [Array<Symbol>] The input vars to use for the prompt. Defaults to [:input]
+    # @param other_inputs [Array<Symbol>] The other input vars to use for the prompt. Defaults to []
+    # @param output_variables [Array<Symbol>] The output vars to use for the prompt. Defaults to [:output]
+    def initialize(template:, input_variables: nil, other_inputs: nil, output_variables: nil)
       @template = template
-      @input_variables = input_variables
-      @output_variables = output_variables
+      @input_variables = input_variables || [:input]
+      @other_inputs = other_inputs || []
+      @output_variables = output_variables || [:output]
     end
 
     # format the prompt with the input variables
+    # @param inputs [Hash] The inputs to use for the prompt.
+    # @return [String] The formatted prompt.
+    # @raise [Boxcars::KeyError] if the template has extra keys.
     def format(inputs)
       @template % inputs
+    rescue ::KeyError => e
+      first_line = e.message.to_s.split("\n").first
+      Boxcars.error "Missing prompt input key: #{first_line}"
+      raise KeyError, "Prompt format error: #{first_line}"
     end
 
     # check if the template is valid
     def template_is_valid?
-      @template.include?("%<input>s") && @template.include?("%<agent_scratchpad>s")
+      all_vars = (input_variables + other_inputs + output_variables).sort
+      template_vars = @template.scan(/%<(\w+)>s/).flatten.map(&:to_sym).sort
+      all_vars == template_vars
+    end
+
+    # missing variables in the template
+    def missing_variables?(inputs)
+      input_vars = [input_variables, other_inputs].flatten.sort
+      return if inputs.keys.sort == input_vars
+
+      raise ArgumentError, "Missing expected input keys, got: #{inputs.keys}. Expected: #{input_vars}"
     end
 
     # create a prompt template from examples
@@ -29,17 +48,22 @@ module Boxcars
     # @param input_variables [Array<Symbol>] The input variables to use for the prompt.
     # @param example_separator [String] The separator to use between the examples. Defaults to "\n\n"
     # @param prefix [String] The prefix to use for the template. Defaults to ""
-    def self.from_examples(examples:, suffix:, input_variables:, example_separator: "\n\n", prefix: "")
+    def self.from_examples(examples:, suffix:, input_variables:, example_separator: "\n\n", prefix: "", **kwargs)
       template = [prefix, examples, suffix].join(example_separator)
-      Prompt.new(template: template, input_variables: input_variables)
+      other_inputs = kwargs[:other_inputs] || []
+      output_variables = kwargs[:output_variables] || [:output]
+      Prompt.new(template: template, input_variables: input_variables, other_inputs: other_inputs,
+                 output_variables: output_variables)
     end
 
     # create a prompt template from a file
     # @param path [String] The path to the file to use for the template.
-    # @param input_variables [Array<Symbol>] The input variables to use for the prompt.
-    def self.from_file(path:, input_variables:)
+    # @param input_variables [Array<Symbol>] The input variables to use for the prompt. Defaults to [:input]
+    # @param output_variables [Array<Symbol>] The output variables to use for the prompt. Defaults to [:output]
+    def self.from_file(path:, input_variables: nil, other_inputs: nil, output_variables: nil)
       template = File.read(path)
-      Prompt.new(template: template, input_variables: input_variables)
+      Prompt.new(template: template, input_variables: input_variables, other_inputs: other_inputs,
+                 output_variables: output_variables)
     end
   end
 end

data/lib/boxcars/ruby_repl.rb

@@ -6,14 +6,14 @@ module Boxcars
     # Execute ruby code
     # @param code [String] The code to run
     def call(code:)
-      puts "RubyREPL: #{code}".colorize(:yellow)
+      Boxcars.debug "RubyREPL: #{code}", :yellow
       output = ""
       IO.popen("ruby", "r+") do |io|
         io.puts code
         io.close_write
         output = io.read
       end
-      puts "Answer: #{output}".colorize(:yellow, style: :bold)
+      Boxcars.debug "Answer: #{output}", :yellow, style: :bold
       output
     end
 

data/lib/boxcars/train/zero_shot.rb

@@ -19,6 +19,7 @@ module Boxcars
       ... (this Thought/Action/Action Input/Observation sequence can repeat N times)
       Thought: I now know the final answer
       Final Answer: the final answer to the original input question
+      Next Actions: up to three suggested actions for the user to take next
     FINPUT
 
     # default prompt suffix

data/lib/boxcars/train.rb

@@ -16,7 +16,7 @@ module Boxcars
       @name = name || self.class.name
       @return_values = [:output]
       @return_intermediate_steps = kwargs[:return_intermediate_steps] || false
-      @max_iterations = kwargs[:max_iterations]
+      @max_iterations = kwargs[:max_iterations] || 25
       @early_stopping_method = kwargs[:early_stopping_method] || "force"
 
       super(prompt: prompt, engine: engine, name: kwargs[:name], description: kwargs[:description])
@@ -112,7 +112,7 @@ module Boxcars
     # @param intermediate_steps [Array<Hash>] The intermediate steps.
     # @return [Hash] The final output.
     def pre_return(output, intermediate_steps)
-      puts output.log.colorize(:yellow, style: :bold)
+      Boxcars.debug output.log, :yellow, style: :bold
       final_output = output.return_values
       final_output["intermediate_steps"] = intermediate_steps if return_intermediate_steps
       final_output
@@ -137,8 +137,6 @@ module Boxcars
         case prompt
         when Prompt
           prompt.template += "\n%<agent_scratchpad>s"
-        # when FewShotPromptTemplate
-        #   prompt.suffix += "\n%<agent_scratchpad>s"
         else
           raise ValueError, "Got unexpected prompt type #{type(prompt)}"
         end
@@ -198,14 +196,14 @@ module Boxcars
             observation = boxcar.run(output.boxcar_input)
             return_direct = boxcar.return_direct
           rescue StandardError => e
-            puts "Error in #{boxcar.name} boxcar#call: #{e}".colorize(:red)
-            raise e
+            error "Error in #{boxcar.name} boxcar#call: #{e}", :red
+            observation = "Error - #{e}, correct and try again."
           end
         else
           observation = "#{output.boxcar} is not a valid boxcar, try another one."
           return_direct = false
         end
-        puts "Observation: #{observation}".colorize(:green)
+        Boxcars.debug "Observation: #{observation}", :green
         intermediate_steps.append([output, observation])
         if return_direct
           output = TrainFinish.new({ return_values[0] => observation }, "")

data/lib/boxcars/version.rb

@@ -2,5 +2,5 @@
 
 module Boxcars
   # The current version of the gem.
-  VERSION = "0.1.6"
+  VERSION = "0.1.8"
 end

data/lib/boxcars.rb

@@ -19,22 +19,8 @@ module Boxcars
   # Error class for all Boxcars security errors.
   class SecurityError < Error; end
 
-  # simple string colorization
-  class ::String
-    # colorize a string
-    # @param color [Symbol] The color to use.
-    # @param options [Hash] The options to use.
-    def colorize(color, options = {})
-      background = options[:background] || options[:bg] || false
-      style = options[:style]
-      offsets = %i[gray red green yellow blue magenta cyan white]
-      styles = %i[normal bold dark italic underline xx xx underline xx strikethrough]
-      start = background ? 40 : 30
-      color_code = start + (offsets.index(color) || 8)
-      style_code = styles.index(style) || 0
-      "\e[#{style_code};#{color_code}m#{self}\e[0m"
-    end
-  end
+  # Error class for all Boxcars key errors.
+  class KeyError < Error; end
 
   # Configuration contains gem settings
   class Configuration
@@ -44,7 +30,6 @@ module Boxcars
     def initialize
       @organization_id = nil
       @logger = Rails.logger if defined?(Rails)
-      @logger ||= Logger.new($stdout)
       @log_prompts = false
     end
 
@@ -68,18 +53,18 @@ module Boxcars
     end
 
     def key_lookup(key, kwargs)
-      rv = if kwargs.key?(key) && !kwargs[key].nil?
-             # override with kwargs if present
-             kwargs[key]
-           elsif (set_val = instance_variable_get("@#{key}"))
-             # use saved value if present
-             set_val
-           else
-             # otherwise, dig out of the environment
-             new_key = ENV.fetch(key.to_s.upcase, nil)
-             new_key
-           end
-      check_key(key, rv)
+      val = if kwargs.key?(key) && !kwargs[key].nil?
+              # override with kwargs if present
+              kwargs[key]
+            elsif (provided_val = instance_variable_get("@#{key}"))
+              # use saved value if present. Set using Boxcars::configuration.the_key = "abcde"
+              provided_val
+            else
+              # otherwise, dig out of the environment
+              env_val = ENV.fetch(key.to_s.upcase, nil)
+              env_val
+            end
+      check_key(key, val)
     end
   end
 
@@ -115,6 +100,59 @@ module Boxcars
       answer.downcase == 'y'
     end
   end
+
+  # Return a logger, possibly if set.
+  def self.logger
+    Boxcars.configuration.logger
+  end
+
+  # Logging system
+  def self.debug(msg, color = nil, **options)
+    msg = colorize(msg.to_s, color, **options) if color
+    if logger
+      logger.debug(msg)
+    else
+      puts msg
+    end
+  end
+
+  def self.info(msg, color = nil, **options)
+    msg = colorize(msg.to_s, color, **options) if color
+    if logger
+      logger.info(msg)
+    else
+      puts msg
+    end
+  end
+
+  def self.warn(msg, color = nil, **options)
+    msg = colorize(msg.to_s, color, **options) if color
+    if logger
+      logger.warn(msg)
+    else
+      puts msg
+    end
+  end
+
+  def self.error(msg, color = nil, **options)
+    msg = colorize(msg.to_s, color, **options) if color
+    if logger
+      logger.error(msg)
+    else
+      puts msg
+    end
+  end
+
+  def self.colorize(str, color, **options)
+    background = options[:background] || options[:bg] || false
+    style = options[:style]
+    offsets = %i[gray red green yellow blue magenta cyan white]
+    styles = %i[normal bold dark italic underline xx xx underline xx strikethrough]
+    start = background ? 40 : 30
+    color_code = start + (offsets.index(color) || 8)
+    style_code = styles.index(style) || 0
+    "\e[#{style_code};#{color_code}m#{str}\e[0m"
+  end
 end
 
 require "boxcars/version"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: boxcars
 version: !ruby/object:Gem::Version
-  version: 0.1.6
+  version: 0.1.8
 platform: ruby
 authors:
 - Francis Sullivan
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2023-02-24 00:00:00.000000000 Z
+date: 2023-03-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: debug
@@ -89,6 +89,7 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- ".env_sample"
 - ".rspec"
 - ".rubocop.yml"
 - CHANGELOG.md