boxcars 0.4.1 → 0.4.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +18 -0
- data/Gemfile.lock +7 -5
- data/README.md +8 -8
- data/lib/boxcars/boxcar/google_search.rb +3 -2
- data/lib/boxcars/boxcar/xml_engine_boxcar.rb +33 -0
- data/lib/boxcars/boxcar.rb +1 -0
- data/lib/boxcars/version.rb +1 -1
- data/run.json +62 -0
- metadata +4 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2e58db8383158eb46fa9b359ca29822796f46a83e9770ab262af147f2f26516a
|
|
4
|
+
data.tar.gz: 261d53fe053d0c3ea7756ee154ff73a077aa5cd2307056f807c20772f06d97bc
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a6629134fb279e4d2556b2a1ec0a1ef43932370099f92b667a8ea45bf355aa9e9dbf5f8ee53cca0c66ba1b09a1198d38cbf168ecb3b7541814d1513287e4f353
|
|
7
|
+
data.tar.gz: fc6d01c0497346f7965296f4ebb7c6b69d67d48f4ec4853a1a8235134e50a966f13cd39ef0b488bab4d27caa8da6d907ae885e70fe63a46732fc859c11c5acde
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,23 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [v0.4.2](https://github.com/BoxcarsAI/boxcars/tree/v0.4.2) (2023-08-05)
|
|
4
|
+
|
|
5
|
+
[Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.4.1...v0.4.2)
|
|
6
|
+
|
|
7
|
+
**Merged pull requests:**
|
|
8
|
+
|
|
9
|
+
- Add xml engine boxcar [\#108](https://github.com/BoxcarsAI/boxcars/pull/108) ([francis](https://github.com/francis))
|
|
10
|
+
- Updated README.md [\#107](https://github.com/BoxcarsAI/boxcars/pull/107) ([Flummoxsoftly](https://github.com/Flummoxsoftly))
|
|
11
|
+
|
|
12
|
+
## [v0.4.1](https://github.com/BoxcarsAI/boxcars/tree/v0.4.1) (2023-07-25)
|
|
13
|
+
|
|
14
|
+
[Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.4.0...v0.4.1)
|
|
15
|
+
|
|
16
|
+
**Merged pull requests:**
|
|
17
|
+
|
|
18
|
+
- Do not use the engine\_prefix to start the LLM prompt for the XML Train [\#106](https://github.com/BoxcarsAI/boxcars/pull/106) ([francis](https://github.com/francis))
|
|
19
|
+
- Adding 16k context [\#105](https://github.com/BoxcarsAI/boxcars/pull/105) ([eltoob](https://github.com/eltoob))
|
|
20
|
+
|
|
3
21
|
## [v0.4.0](https://github.com/BoxcarsAI/boxcars/tree/v0.4.0) (2023-07-19)
|
|
4
22
|
|
|
5
23
|
[Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.3.5...v0.4.0)
|
data/Gemfile.lock
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
boxcars (0.4.
|
|
4
|
+
boxcars (0.4.3)
|
|
5
5
|
anthropic (~> 0.1)
|
|
6
6
|
google_search_results (~> 2.2)
|
|
7
7
|
gpt4all (~> 0.0.4)
|
|
@@ -48,6 +48,7 @@ GEM
|
|
|
48
48
|
async
|
|
49
49
|
async-pool (0.4.0)
|
|
50
50
|
async (>= 1.25)
|
|
51
|
+
base64 (0.1.1)
|
|
51
52
|
concurrent-ruby (1.2.2)
|
|
52
53
|
console (1.17.4)
|
|
53
54
|
fiber-annotation
|
|
@@ -61,7 +62,8 @@ GEM
|
|
|
61
62
|
domain_name (0.5.20190701)
|
|
62
63
|
unf (>= 0.0.5, < 1.0.0)
|
|
63
64
|
dotenv (2.8.1)
|
|
64
|
-
faraday (2.7.
|
|
65
|
+
faraday (2.7.11)
|
|
66
|
+
base64
|
|
65
67
|
faraday-net_http (>= 2.0, < 3.1)
|
|
66
68
|
ruby2_keywords (>= 0.0.4)
|
|
67
69
|
faraday-http-cache (2.5.0)
|
|
@@ -107,9 +109,9 @@ GEM
|
|
|
107
109
|
multipart-post (2.3.0)
|
|
108
110
|
netrc (0.11.0)
|
|
109
111
|
nio4r (2.5.9)
|
|
110
|
-
nokogiri (1.15.
|
|
112
|
+
nokogiri (1.15.4-arm64-darwin)
|
|
111
113
|
racc (~> 1.4)
|
|
112
|
-
nokogiri (1.15.
|
|
114
|
+
nokogiri (1.15.4-x86_64-linux)
|
|
113
115
|
racc (~> 1.4)
|
|
114
116
|
octokit (4.25.1)
|
|
115
117
|
faraday (>= 1, < 3)
|
|
@@ -177,7 +179,7 @@ GEM
|
|
|
177
179
|
rubocop (~> 1.33)
|
|
178
180
|
rubocop-capybara (~> 2.17)
|
|
179
181
|
rubocop-factory_bot (~> 2.22)
|
|
180
|
-
ruby-openai (4.2
|
|
182
|
+
ruby-openai (4.3.2)
|
|
181
183
|
faraday (>= 1)
|
|
182
184
|
faraday-multipart (>= 1)
|
|
183
185
|
ruby-progressbar (1.13.0)
|
data/README.md
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
<a href="https://github.com/BoxcarsAI/boxcars/blob/main/LICENSE.txt"><img src="https://img.shields.io/badge/license-MIT-informational" alt="License"></a>
|
|
11
11
|
</p>
|
|
12
12
|
|
|
13
|
-
Boxcars is a gem that enables you to create new systems with AI composability, using various concepts such as LLMs (OpenAI, Anthropic, Gpt4all), Search, SQL (with both Sequel
|
|
13
|
+
Boxcars is a gem that enables you to create new systems with AI composability, using various concepts such as LLMs (OpenAI, Anthropic, Gpt4all), Search, SQL (with both Sequel and Active Record support), Rails Active Record, Vector Search and more. This can even be extended with your concepts as well (including your concepts).
|
|
14
14
|
|
|
15
15
|
This gem was inspired by the popular Python library Langchain. However, we wanted to give it a Ruby spin and make it more user-friendly for beginners to get started.
|
|
16
16
|
|
|
@@ -48,7 +48,7 @@ Or install it yourself as:
|
|
|
48
48
|
|
|
49
49
|
We will be adding more examples soon, but here are a couple to get you started. First, you'll need to set up your environment variables for OpenAI and Google SERP (OPENAI_ACCESS_TOKEN, SERPAPI_API_KEY). If you prefer not to set these variables in your environment, you can pass them directly into the API.
|
|
50
50
|
|
|
51
|
-
In the examples below, we added one
|
|
51
|
+
In the examples below, we added one Ruby gem to load the environment at the first line, but depending on what you want, you might not need this.
|
|
52
52
|
```ruby
|
|
53
53
|
require "dotenv/load"
|
|
54
54
|
require "boxcars"
|
|
@@ -68,12 +68,12 @@ irb -r dotenv/load -r ./lib/boxcars
|
|
|
68
68
|
# run the calculator
|
|
69
69
|
engine = Boxcars::Openai.new(max_tokens: 256)
|
|
70
70
|
calc = Boxcars::Calculator.new(engine: engine)
|
|
71
|
-
puts calc.run "what is pi to the
|
|
71
|
+
puts calc.run "what is pi to the fourth power divided by 22.1?"
|
|
72
72
|
```
|
|
73
73
|
Produces:
|
|
74
74
|
```text
|
|
75
75
|
> Entering Calculator#run
|
|
76
|
-
what is pi to the
|
|
76
|
+
what is pi to the fourth power divided by 22.1?
|
|
77
77
|
RubyREPL: puts (Math::PI**4)/22.1
|
|
78
78
|
Answer: 4.407651178009159
|
|
79
79
|
|
|
@@ -82,20 +82,20 @@ Answer: 4.407651178009159
|
|
|
82
82
|
4.407651178009159
|
|
83
83
|
```
|
|
84
84
|
|
|
85
|
-
Note that since Openai is currently the most used Engine, if you do not pass in an engine, it will default as expected. So, this is the
|
|
85
|
+
Note that since Openai is currently the most used Engine, if you do not pass in an engine, it will default as expected. So, this is the equivalent shorter version of the above script:
|
|
86
86
|
```ruby
|
|
87
87
|
# run the calculator
|
|
88
88
|
calc = Boxcars::Calculator.new # just use the default Engine
|
|
89
|
-
puts calc.run "what is pi to the
|
|
89
|
+
puts calc.run "what is pi to the fourth power divided by 22.1?"
|
|
90
90
|
```
|
|
91
91
|
You can change the default_engine with `Boxcars::configuration.default_engine = NewDefaultEngine`
|
|
92
92
|
### Boxcars currently implemented
|
|
93
93
|
|
|
94
94
|
Here is what we have so far, but please put up a PR with your new ideas.
|
|
95
|
-
- GoogleSearch: uses the SERP API to do
|
|
95
|
+
- GoogleSearch: uses the SERP API to do searches
|
|
96
96
|
- WikipediaSearch: uses the Wikipedia API to do searches
|
|
97
97
|
- Calculator: uses an Engine to generate ruby code to do math
|
|
98
|
-
- SQL: given an ActiveRecord connection, it will generate and run sql
|
|
98
|
+
- SQL: given an ActiveRecord connection, it will generate and run sql statements from a prompt.
|
|
99
99
|
- ActiveRecord: given an ActiveRecord connection, it will generate and run ActiveRecord statements from a prompt.
|
|
100
100
|
- Swagger: give a Swagger Open API file (YAML or JSON), answer questions about or run against the referenced service. See [here](https://github.com/BoxcarsAI/boxcars/blob/main/notebooks/swagger_examples.ipynb) for examples.
|
|
101
101
|
|
|
@@ -50,11 +50,12 @@ module Boxcars
|
|
|
50
50
|
%i[sports_results game_spotlight],
|
|
51
51
|
%i[knowledge_graph description],
|
|
52
52
|
[:organic_results, 0, :snippet],
|
|
53
|
-
[:organic_results, 0, :snippet_highlighted_words, 0]
|
|
53
|
+
[:organic_results, 0, :snippet_highlighted_words, 0],
|
|
54
|
+
[:organic_results, 0, :title]
|
|
54
55
|
].freeze
|
|
55
56
|
|
|
56
57
|
def find_answer(res)
|
|
57
|
-
raise Error, "Got error from SerpAPI: {res[:error]}" if res[:error]
|
|
58
|
+
raise Error, "Got error from SerpAPI: #{res[:error]}" if res[:error]
|
|
58
59
|
|
|
59
60
|
ANSWER_LOCATIONS.each do |path|
|
|
60
61
|
next unless res.dig(*path)
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Boxcars is a framework for running a series of tools to get an answer to a question.
|
|
4
|
+
module Boxcars
|
|
5
|
+
# For Boxcars that use an engine to do their work.
|
|
6
|
+
class XMLEngineBoxcar < EngineBoxcar
|
|
7
|
+
# An XML Engine Boxcar is a container for a single tool to run.
|
|
8
|
+
|
|
9
|
+
# Parse out the action and input from the engine output.
|
|
10
|
+
# @param engine_output [String] The output from the engine.
|
|
11
|
+
# @return [Array<String>] The action and input.
|
|
12
|
+
def get_answer(engine_output)
|
|
13
|
+
xn_get_answer(XNode.from_xml(engine_output))
|
|
14
|
+
rescue StandardError => e
|
|
15
|
+
Result.from_error("Error: #{e.message}:\n#{engine_output}")
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
# get answer an XNode
|
|
19
|
+
# @param xnode [XNode] The XNode to use.
|
|
20
|
+
# @return [Array<String, String>] The action and input.
|
|
21
|
+
def xn_get_answer(xnode)
|
|
22
|
+
reply = xnode.xtext("//reply")
|
|
23
|
+
|
|
24
|
+
if reply.present?
|
|
25
|
+
Result.new(status: :ok, answer: reply, explanation: reply)
|
|
26
|
+
else
|
|
27
|
+
# we have an unexpected output from the engine
|
|
28
|
+
Result.new(status: :error, answer: nil,
|
|
29
|
+
explanation: "You gave me an improperly formatted answer or didn't use tags. I was expecting a reply.")
|
|
30
|
+
end
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
end
|
data/lib/boxcars/boxcar.rb
CHANGED
|
@@ -211,6 +211,7 @@ end
|
|
|
211
211
|
require "boxcars/observation"
|
|
212
212
|
require "boxcars/result"
|
|
213
213
|
require "boxcars/boxcar/engine_boxcar"
|
|
214
|
+
require "boxcars/boxcar/xml_engine_boxcar"
|
|
214
215
|
require "boxcars/boxcar/calculator"
|
|
215
216
|
require "boxcars/boxcar/ruby_calculator"
|
|
216
217
|
require "boxcars/boxcar/google_search"
|
data/lib/boxcars/version.rb
CHANGED
data/run.json
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
<output>
|
|
2
|
+
<articles>
|
|
3
|
+
<article id="46">
|
|
4
|
+
<confidence>
|
|
5
|
+
<cscore>4</cscore>
|
|
6
|
+
<creason>Article discusses a solar panel manufacturing project and related tax incentives in Fort Mill, South Carolina</creason>
|
|
7
|
+
</confidence>
|
|
8
|
+
<favorability>
|
|
9
|
+
<fscore>2</fscore>
|
|
10
|
+
<freason>Article suggests some uncertainty around whether the tax incentives will be approved</freason>
|
|
11
|
+
</favorability>
|
|
12
|
+
<county>York County</county>
|
|
13
|
+
<state>South Carolina</state>
|
|
14
|
+
</article>
|
|
15
|
+
|
|
16
|
+
<article id="47">
|
|
17
|
+
<confidence>
|
|
18
|
+
<cscore>5</cscore>
|
|
19
|
+
<creason>Article clearly discusses a solar energy project in Morgan County</creason>
|
|
20
|
+
</confidence>
|
|
21
|
+
<favorability>
|
|
22
|
+
<fscore>3</fscore>
|
|
23
|
+
<freason>Article suggests some public concerns but does not take a strong stance</freason>
|
|
24
|
+
</favorability>
|
|
25
|
+
<county>Morgan County</county>
|
|
26
|
+
<state>Colorado</state>
|
|
27
|
+
</article>
|
|
28
|
+
|
|
29
|
+
<article id="48">
|
|
30
|
+
<confidence>
|
|
31
|
+
<cscore>5</cscore>
|
|
32
|
+
<creason>Article discusses regulations for large-scale solar projects in Linn County, Iowa</creason>
|
|
33
|
+
</confidence>
|
|
34
|
+
<favorability>
|
|
35
|
+
<fscore>2</fscore>
|
|
36
|
+
<freason>Article suggests the county is moving toward stricter regulations</freason>
|
|
37
|
+
</favorability>
|
|
38
|
+
<county>Linn County</county>
|
|
39
|
+
<state>Iowa</state>
|
|
40
|
+
</article>
|
|
41
|
+
|
|
42
|
+
<article id="49">
|
|
43
|
+
<confidence>
|
|
44
|
+
<cscore>5</cscore>
|
|
45
|
+
<creason>Article clearly discusses approval of a solar project expansion in Clinton County</creason>
|
|
46
|
+
</confidence>
|
|
47
|
+
<favorability>
|
|
48
|
+
<fscore>4</fscore>
|
|
49
|
+
<freason>Article suggests consensus in approving the expansion</freason>
|
|
50
|
+
</favorability>
|
|
51
|
+
<county>Clinton County</county>
|
|
52
|
+
<state>Iowa</state>
|
|
53
|
+
</article>
|
|
54
|
+
|
|
55
|
+
<article id="50">
|
|
56
|
+
<confidence>
|
|
57
|
+
<cscore>1</cscore>
|
|
58
|
+
<creason>Article discusses solar panels on a farm building in the UK, not a utility-scale project in the US</creason>
|
|
59
|
+
</article>
|
|
60
|
+
|
|
61
|
+
</articles>
|
|
62
|
+
</output>
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: boxcars
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.4.
|
|
4
|
+
version: 0.4.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Francis Sullivan
|
|
@@ -9,7 +9,7 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: exe
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2023-
|
|
12
|
+
date: 2023-09-19 00:00:00.000000000 Z
|
|
13
13
|
dependencies:
|
|
14
14
|
- !ruby/object:Gem::Dependency
|
|
15
15
|
name: anthropic
|
|
@@ -145,6 +145,7 @@ files:
|
|
|
145
145
|
- lib/boxcars/boxcar/url_text.rb
|
|
146
146
|
- lib/boxcars/boxcar/vector_answer.rb
|
|
147
147
|
- lib/boxcars/boxcar/wikipedia_search.rb
|
|
148
|
+
- lib/boxcars/boxcar/xml_engine_boxcar.rb
|
|
148
149
|
- lib/boxcars/conversation.rb
|
|
149
150
|
- lib/boxcars/conversation_prompt.rb
|
|
150
151
|
- lib/boxcars/engine.rb
|
|
@@ -182,6 +183,7 @@ files:
|
|
|
182
183
|
- lib/boxcars/vector_store/split_text.rb
|
|
183
184
|
- lib/boxcars/version.rb
|
|
184
185
|
- lib/boxcars/x_node.rb
|
|
186
|
+
- run.json
|
|
185
187
|
homepage: https://github.com/BoxcarsAI/boxcars
|
|
186
188
|
licenses:
|
|
187
189
|
- MIT
|