boxcars 0.7.7 → 0.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +6 -3
  3. data/.ruby-version +1 -1
  4. data/CHANGELOG.md +17 -0
  5. data/Gemfile +3 -13
  6. data/Gemfile.lock +30 -25
  7. data/POSTHOG_TEST_README.md +118 -0
  8. data/README.md +305 -0
  9. data/boxcars.gemspec +2 -2
  10. data/lib/boxcars/boxcar/active_record.rb +9 -10
  11. data/lib/boxcars/boxcar/calculator.rb +2 -2
  12. data/lib/boxcars/boxcar/engine_boxcar.rb +4 -4
  13. data/lib/boxcars/boxcar/google_search.rb +2 -2
  14. data/lib/boxcars/boxcar/json_engine_boxcar.rb +1 -1
  15. data/lib/boxcars/boxcar/ruby_calculator.rb +1 -1
  16. data/lib/boxcars/boxcar/sql_base.rb +4 -4
  17. data/lib/boxcars/boxcar/swagger.rb +3 -3
  18. data/lib/boxcars/boxcar/vector_answer.rb +3 -3
  19. data/lib/boxcars/boxcar/xml_engine_boxcar.rb +1 -1
  20. data/lib/boxcars/boxcar.rb +6 -6
  21. data/lib/boxcars/conversation_prompt.rb +3 -3
  22. data/lib/boxcars/engine/anthropic.rb +121 -23
  23. data/lib/boxcars/engine/cerebras.rb +2 -2
  24. data/lib/boxcars/engine/cohere.rb +135 -9
  25. data/lib/boxcars/engine/gemini_ai.rb +151 -76
  26. data/lib/boxcars/engine/google.rb +2 -2
  27. data/lib/boxcars/engine/gpt4all_eng.rb +92 -34
  28. data/lib/boxcars/engine/groq.rb +124 -73
  29. data/lib/boxcars/engine/intelligence_base.rb +52 -17
  30. data/lib/boxcars/engine/ollama.rb +127 -47
  31. data/lib/boxcars/engine/openai.rb +186 -103
  32. data/lib/boxcars/engine/perplexityai.rb +116 -136
  33. data/lib/boxcars/engine/together.rb +2 -2
  34. data/lib/boxcars/engine/unified_observability.rb +430 -0
  35. data/lib/boxcars/engine.rb +4 -3
  36. data/lib/boxcars/engines.rb +74 -0
  37. data/lib/boxcars/observability.rb +44 -0
  38. data/lib/boxcars/observability_backend.rb +17 -0
  39. data/lib/boxcars/observability_backends/multi_backend.rb +42 -0
  40. data/lib/boxcars/observability_backends/posthog_backend.rb +89 -0
  41. data/lib/boxcars/observation.rb +8 -8
  42. data/lib/boxcars/prompt.rb +16 -4
  43. data/lib/boxcars/result.rb +7 -12
  44. data/lib/boxcars/ruby_repl.rb +1 -1
  45. data/lib/boxcars/train/train_action.rb +1 -1
  46. data/lib/boxcars/train/xml_train.rb +3 -3
  47. data/lib/boxcars/train/xml_zero_shot.rb +1 -1
  48. data/lib/boxcars/train/zero_shot.rb +3 -3
  49. data/lib/boxcars/train.rb +1 -1
  50. data/lib/boxcars/vector_search.rb +5 -5
  51. data/lib/boxcars/vector_store/pgvector/build_from_array.rb +115 -88
  52. data/lib/boxcars/vector_store/pgvector/build_from_files.rb +105 -80
  53. data/lib/boxcars/vector_store/pgvector/save_to_database.rb +147 -122
  54. data/lib/boxcars/vector_store/pgvector/search.rb +156 -131
  55. data/lib/boxcars/vector_store.rb +4 -4
  56. data/lib/boxcars/version.rb +1 -1
  57. data/lib/boxcars.rb +31 -20
  58. metadata +25 -21
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8c0b63da11bca5d10f8de95aa51f42d6acea996881e636e3006b98337f01e564
4
- data.tar.gz: 63d7554f52d4c29f0e58511c827f490f38215b26b45003413309f12df150e534
3
+ metadata.gz: e76c5930ad4510169e7b27bca16789e5e8b72f1627f8e05546fe6450a649416a
4
+ data.tar.gz: 7a3a1f3e8eb58674481f57e814b68d797135f5ea41856629bbbae9f17743e429
5
5
  SHA512:
6
- metadata.gz: da06e0a21403d259342a5e516519bed361328737ab3601a262a926009ea5199e26d36d0a3a52b77f7089f016741bc991bc46c605f6502d17fcdc48cedb9f10fa
7
- data.tar.gz: 828c9799e4a9405a72b2975819f70656c7fb284868f103b0ea090e27319d1a54c828b51f7af6ebdb11cbe0b2028f5c2ff691a70b8bbc6c9426f88f6f0cabe5ba
6
+ metadata.gz: 0b660877e4644a88e4a024512be55b2eeb5ea5cae8fb72a66f117a7ebfca2eefce6f2cfc10587afd9083b03b40e38c0fb30ca56ffbd4ce6723c09f2bf886acdf
7
+ data.tar.gz: f17d7b1cc975749780feecf2dfca65056332e98bb6b2a3ef40cbe0663820ed66ee51f88776910590b8c27a8ec4fcfe586c188ee7126c99127e4dd410d4e7f8fb
data/.rubocop.yml CHANGED
@@ -6,7 +6,7 @@ plugins:
6
6
 
7
7
 
8
8
  AllCops:
9
- TargetRubyVersion: 3
9
+ TargetRubyVersion: 3.2
10
10
  Exclude:
11
11
  - 'bin/{rails,rake}'
12
12
  - 'node_modules/**/*'
@@ -72,7 +72,7 @@ Metrics/BlockLength:
72
72
  - 'spec/**/*'
73
73
 
74
74
  Metrics/MethodLength:
75
- Max: 35
75
+ Max: 40
76
76
  Exclude:
77
77
  - 'spec/**/*'
78
78
 
@@ -95,7 +95,7 @@ Naming/AccessorMethodName:
95
95
  Enabled: false
96
96
 
97
97
  Metrics/PerceivedComplexity:
98
- Max: 10
98
+ Max: 20
99
99
 
100
100
  Style/RescueStandardError:
101
101
  Enabled: false
@@ -171,3 +171,6 @@ RSpec/ExampleLength:
171
171
 
172
172
  RSpec/MultipleExpectations:
173
173
  Enabled: false
174
+
175
+ RSpec/BeforeAfterAll:
176
+ Enabled: false
data/.ruby-version CHANGED
@@ -1 +1 @@
1
- 3.3.5
1
+ 3.4.4
data/CHANGELOG.md CHANGED
@@ -1,5 +1,22 @@
1
1
  # Changelog
2
2
 
3
+ ## [v0.8.0](https://github.com/BoxcarsAI/boxcars/tree/v0.8.0) (2025-06-03)
4
+
5
+ [Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.7.7...v0.8.0)
6
+
7
+ **Closed issues:**
8
+
9
+ - ActiveRecord translations [\#249](https://github.com/BoxcarsAI/boxcars/issues/249)
10
+ - Any chance of getting AWS Bedrock as a provider? [\#239](https://github.com/BoxcarsAI/boxcars/issues/239)
11
+
12
+ **Merged pull requests:**
13
+
14
+ - Add observability hooks [\#251](https://github.com/BoxcarsAI/boxcars/pull/251) ([francis](https://github.com/francis))
15
+
16
+ ## [v0.7.7](https://github.com/BoxcarsAI/boxcars/tree/v0.7.7) (2025-04-23)
17
+
18
+ [Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.7.6...v0.7.7)
19
+
3
20
  ## [v0.7.6](https://github.com/BoxcarsAI/boxcars/tree/v0.7.6) (2025-04-22)
4
21
 
5
22
  [Full Changelog](https://github.com/BoxcarsAI/boxcars/compare/v0.7.5...v0.7.6)
data/Gemfile CHANGED
@@ -5,36 +5,26 @@ source "https://rubygems.org"
5
5
  # Specify your gem's dependencies in boxcars.gemspec
6
6
  gemspec
7
7
 
8
- gem "debug", "~> 1.9"
9
-
10
8
  gem "dotenv", "~> 3.1"
11
-
12
9
  gem "rake", "~> 13.2"
13
-
14
10
  gem "sqlite3", "~> 2.0"
15
-
16
11
  gem "async", "~>1.32.1"
17
-
18
12
  gem "activerecord", "~> 7.1"
19
-
20
- gem "github_changelog_generator", "~> 1.16"
21
-
22
13
  gem "faraday-retry", "~> 2.0"
23
-
24
14
  gem "activesupport", "~> 7.1"
25
-
26
15
  gem "rest-client", "~> 2.1"
27
-
28
16
  gem "hnswlib", "~> 0.9.0"
29
-
30
17
  gem "pg", "~> 1.5"
31
18
  gem "pgvector", "~> 0.2.2"
32
19
 
33
20
  group :development, :test do
21
+ gem "debug", "~> 1.9"
34
22
  gem "rspec", "~> 3.13"
35
23
  gem "rubocop", "~> 1.67"
36
24
  gem "vcr", "~> 6.3.1"
37
25
  gem "webmock", "~> 3.24.0"
38
26
  gem "rubocop-rake", "~> 0.6.0"
39
27
  gem "rubocop-rspec", "~> 3.2"
28
+ gem "posthog-ruby", require: false
29
+ gem "github_changelog_generator", "~> 1.16", require: false
40
30
  end
data/Gemfile.lock CHANGED
@@ -1,13 +1,13 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- boxcars (0.7.7)
4
+ boxcars (0.8.1)
5
+ faraday-retry (~> 2.0)
5
6
  google_search_results (~> 2.2)
6
7
  gpt4all (~> 0.0.5)
7
8
  hnswlib (~> 0.9)
8
9
  intelligence (>= 0.8)
9
10
  nokogiri (~> 1.18)
10
- pgvector (~> 0.2)
11
11
  ruby-anthropic (~> 0.4)
12
12
  ruby-openai (>= 7.3)
13
13
 
@@ -54,11 +54,11 @@ GEM
54
54
  async
55
55
  async-pool (0.10.3)
56
56
  async (>= 1.25)
57
- base64 (0.2.0)
58
- benchmark (0.4.0)
59
- bigdecimal (3.1.9)
57
+ base64 (0.3.0)
58
+ benchmark (0.4.1)
59
+ bigdecimal (3.2.1)
60
60
  concurrent-ruby (1.3.5)
61
- connection_pool (2.5.1)
61
+ connection_pool (2.5.3)
62
62
  console (1.30.2)
63
63
  fiber-annotation
64
64
  fiber-local (~> 1.1)
@@ -70,13 +70,14 @@ GEM
70
70
  debug (1.10.0)
71
71
  irb (~> 1.10)
72
72
  reline (>= 0.3.8)
73
- diff-lcs (1.6.1)
73
+ diff-lcs (1.6.2)
74
74
  domain_name (0.6.20240107)
75
75
  dotenv (3.1.8)
76
- drb (2.2.1)
76
+ drb (2.2.3)
77
77
  dynamicschema (1.0.0.beta04)
78
+ erb (5.0.1)
78
79
  event_stream_parser (1.0.0)
79
- faraday (2.13.0)
80
+ faraday (2.13.1)
80
81
  faraday-net_http (>= 2.0, < 3.5)
81
82
  json
82
83
  logger
@@ -106,7 +107,7 @@ GEM
106
107
  faraday (~> 2.7)
107
108
  os (~> 1.1)
108
109
  tty-progressbar (~> 0.18.2)
109
- hashdiff (1.1.2)
110
+ hashdiff (1.2.0)
110
111
  hnswlib (0.9.0)
111
112
  http-accept (1.7.0)
112
113
  http-cookie (1.0.8)
@@ -123,15 +124,15 @@ GEM
123
124
  pp (>= 0.6.0)
124
125
  rdoc (>= 4.0.0)
125
126
  reline (>= 0.4.2)
126
- json (2.10.2)
127
+ json (2.12.2)
127
128
  json-repair (0.2.0)
128
- language_server-protocol (3.17.0.4)
129
+ language_server-protocol (3.17.0.5)
129
130
  lint_roller (1.1.0)
130
131
  logger (1.7.0)
131
- mime-types (3.6.2)
132
+ mime-types (3.7.0)
132
133
  logger
133
- mime-types-data (~> 3.2015)
134
- mime-types-data (3.2025.0415)
134
+ mime-types-data (~> 3.2025, >= 3.2025.0507)
135
+ mime-types-data (3.2025.0527)
135
136
  minitest (5.25.5)
136
137
  multi_json (1.15.0)
137
138
  multipart-post (2.4.1)
@@ -165,6 +166,8 @@ GEM
165
166
  racc
166
167
  pg (1.5.9)
167
168
  pgvector (0.2.2)
169
+ posthog-ruby (3.0.1)
170
+ concurrent-ruby (~> 1)
168
171
  pp (0.6.2)
169
172
  prettyprint
170
173
  prettyprint (0.2.0)
@@ -176,14 +179,15 @@ GEM
176
179
  protocol-http2 (0.16.0)
177
180
  protocol-hpack (~> 1.4)
178
181
  protocol-http (~> 0.18)
179
- psych (5.2.3)
182
+ psych (5.2.6)
180
183
  date
181
184
  stringio
182
- public_suffix (6.0.1)
185
+ public_suffix (6.0.2)
183
186
  racc (1.8.1)
184
187
  rainbow (3.1.1)
185
- rake (13.2.1)
186
- rdoc (6.13.1)
188
+ rake (13.3.0)
189
+ rdoc (6.14.0)
190
+ erb
187
191
  psych (>= 4.0.0)
188
192
  regexp_parser (2.10.0)
189
193
  reline (0.6.1)
@@ -194,20 +198,20 @@ GEM
194
198
  mime-types (>= 1.16, < 4.0)
195
199
  netrc (~> 0.8)
196
200
  rexml (3.4.1)
197
- rspec (3.13.0)
201
+ rspec (3.13.1)
198
202
  rspec-core (~> 3.13.0)
199
203
  rspec-expectations (~> 3.13.0)
200
204
  rspec-mocks (~> 3.13.0)
201
- rspec-core (3.13.3)
205
+ rspec-core (3.13.4)
202
206
  rspec-support (~> 3.13.0)
203
- rspec-expectations (3.13.3)
207
+ rspec-expectations (3.13.5)
204
208
  diff-lcs (>= 1.2.0, < 2.0)
205
209
  rspec-support (~> 3.13.0)
206
- rspec-mocks (3.13.2)
210
+ rspec-mocks (3.13.5)
207
211
  diff-lcs (>= 1.2.0, < 2.0)
208
212
  rspec-support (~> 3.13.0)
209
- rspec-support (3.13.2)
210
- rubocop (1.75.2)
213
+ rspec-support (3.13.4)
214
+ rubocop (1.75.8)
211
215
  json (~> 2.3)
212
216
  language_server-protocol (~> 3.17.0.2)
213
217
  lint_roller (~> 1.1.0)
@@ -292,6 +296,7 @@ DEPENDENCIES
292
296
  hnswlib (~> 0.9.0)
293
297
  pg (~> 1.5)
294
298
  pgvector (~> 0.2.2)
299
+ posthog-ruby
295
300
  rake (~> 13.2)
296
301
  rest-client (~> 2.1)
297
302
  rspec (~> 3.13)
@@ -0,0 +1,118 @@
1
+ # Boxcars Engines PostHog Observability Test
2
+
3
+ This test program demonstrates how to use the Boxcars library with PostHog observability backend to track AI engine usage.
4
+
5
+ ## Prerequisites
6
+
7
+ 1. **PostHog Ruby Gem**: Install the required gem
8
+ ```bash
9
+ gem install posthog-ruby
10
+ ```
11
+
12
+ 2. **Environment Variables**: Ensure your `.env` file contains:
13
+ ```
14
+ POSTHOG_API_KEY=your_posthog_project_api_key
15
+ POSTHOG_HOST=https://app.posthog.com # or your self-hosted instance
16
+ ```
17
+
18
+ 3. **AI Provider API Keys**: For testing different engines, you'll need:
19
+ ```
20
+ openai_access_token=your_openai_token
21
+ GOOGLE_API_KEY=your_google_api_key
22
+ ANTHROPIC_API_KEY=your_anthropic_key
23
+ GROQ_API_KEY=your_groq_key
24
+ ```
25
+
26
+ ## Usage
27
+
28
+ ### Method 1: IRB Interactive Session (Recommended)
29
+
30
+ Start IRB with the required dependencies:
31
+
32
+ ```bash
33
+ irb -r dotenv/load -r boxcars -r debug -r boxcars/observability_backends/posthog_backend
34
+ ```
35
+
36
+ Then in the IRB session:
37
+
38
+ ```ruby
39
+ # Load and run the test
40
+ load 'test_engines_with_posthog.rb'
41
+
42
+ # Or set up PostHog backend manually:
43
+ Boxcars::Observability.backend = Boxcars::PosthogBackend.new(
44
+ api_key: ENV['POSTHOG_API_KEY'],
45
+ host: ENV['POSTHOG_HOST']
46
+ )
47
+
48
+ # Run manual tests
49
+ manual_test(model: 'gpt-4o', prompt: 'What is machine learning?')
50
+ manual_test(model: 'flash', prompt: 'Explain Ruby in one sentence')
51
+ manual_test(model: 'sonnet', prompt: 'Write a short poem about coding')
52
+ ```
53
+
54
+ ### Method 2: Direct Ruby Execution
55
+
56
+ ```bash
57
+ ruby test_engines_with_posthog.rb
58
+ ```
59
+
60
+ ## What the Test Does
61
+
62
+ 1. **Initializes PostHog Backend**: Sets up the PostHog observability backend with your API credentials
63
+ 2. **Tests Multiple Engines**: Runs tests against various AI engines:
64
+ - Gemini Flash (Default)
65
+ - GPT-4o (OpenAI)
66
+ - Claude Sonnet (Anthropic)
67
+ - Groq Llama
68
+ 3. **Tracks Observability Events**: Each API call generates PostHog events with AI-specific properties
69
+ 4. **Provides Manual Testing**: Includes a `manual_test` function for interactive testing
70
+
71
+ ## PostHog Events
72
+
73
+ The test will generate events in PostHog with properties like:
74
+
75
+ - `$ai_model`: The AI model used (e.g., "gpt-4o", "gemini-2.5-flash")
76
+ - `$ai_provider`: The provider (e.g., "openai", "google", "anthropic")
77
+ - `$ai_input_tokens`: Number of input tokens
78
+ - `$ai_output_tokens`: Number of output tokens
79
+ - `$ai_latency`: Response time in seconds
80
+ - `$ai_http_status`: HTTP status code
81
+ - `$ai_trace_id`: Unique trace identifier
82
+ - `$ai_is_error`: Boolean indicating if there was an error
83
+
84
+ ## Viewing Results
85
+
86
+ After running the test:
87
+
88
+ 1. Go to your PostHog dashboard
89
+ 2. Navigate to Events or Live Events
90
+ 3. Look for events with AI-related properties
91
+ 4. You can create insights and dashboards to analyze AI usage patterns
92
+
93
+ ## Troubleshooting
94
+
95
+ - **Missing PostHog gem**: Install with `gem install posthog-ruby`
96
+ - **Missing API keys**: Check your `.env` file has the required keys
97
+ - **Engine errors**: Some engines may fail if you don't have valid API keys for those providers
98
+ - **No events in PostHog**: Check your PostHog API key and host configuration
99
+
100
+ ## Available Models
101
+
102
+ You can test with these model aliases:
103
+
104
+ - `flash` - Gemini 2.5 Flash
105
+ - `gpt-4o` - OpenAI GPT-4o
106
+ - `sonnet` - Claude Sonnet
107
+ - `groq` - Groq Llama
108
+ - `online` - Perplexity Sonar
109
+ - And many more (see `lib/boxcars/engines.rb`)
110
+
111
+ ## Example Manual Tests
112
+
113
+ ```ruby
114
+ # Test different models
115
+ manual_test(model: 'flash', prompt: 'Explain quantum computing')
116
+ manual_test(model: 'gpt-4o', prompt: 'Write a Python function to sort a list')
117
+ manual_test(model: 'sonnet', prompt: 'What are the benefits of functional programming?')
118
+ manual_test(model: 'groq', prompt: 'Describe the difference between AI and ML')
data/README.md CHANGED
@@ -149,6 +149,311 @@ Also, if you set this flag: `Boxcars.configuration.log_prompts = true`
149
149
  The actual prompts handed to the connected Engine will be logged. This is off by default because it is very wordy, but handy if you are debugging prompts.
150
150
 
151
151
  Otherwise, we print to standard out.
152
+
153
+ ### Engine Factory (Engines)
154
+
155
+ Boxcars provides a convenient factory class `Boxcars::Engines` that simplifies creating engine instances using model names and aliases instead of remembering full class names and model strings.
156
+
157
+ #### Basic Usage
158
+
159
+ ```ruby
160
+ # Using default model (gemini-2.5-flash-preview-05-20)
161
+ engine = Boxcars::Engines.engine
162
+
163
+ # Using specific models with convenient aliases
164
+ gpt_engine = Boxcars::Engines.engine(model: "gpt-4o")
165
+ claude_engine = Boxcars::Engines.engine(model: "sonnet")
166
+ gemini_engine = Boxcars::Engines.engine(model: "flash")
167
+ groq_engine = Boxcars::Engines.engine(model: "groq")
168
+ ```
169
+
170
+ #### Supported Model Aliases
171
+
172
+ **OpenAI Models:**
173
+ - `"gpt-4o"`, `"gpt-3.5-turbo"`, `"o1-preview"` - Creates `Boxcars::Openai` engines
174
+
175
+ **Anthropic Models:**
176
+ - `"anthropic"`, `"sonnet"` - Creates `Boxcars::Anthropic` with Claude Sonnet
177
+ - `"opus"` - Creates `Boxcars::Anthropic` with Claude Opus
178
+ - `"claude-3-5-sonnet"`, etc. - Any model starting with "claude-"
179
+
180
+ **Groq Models:**
181
+ - `"groq"` - Creates `Boxcars::Groq` with Llama 3.3 70B
182
+ - `"deepseek"` - Creates `Boxcars::Groq` with DeepSeek R1
183
+ - `"mistral"` - Creates `Boxcars::Groq` with Mistral
184
+ - Models starting with `"mistral-"`, `"meta-llama/"`, or `"deepseek-"`
185
+
186
+ **Gemini Models:**
187
+ - `"flash"`, `"gemini-flash"` - Creates `Boxcars::GeminiAi` with Gemini 2.5 Flash
188
+ - `"gemini-pro"` - Creates `Boxcars::GeminiAi` with Gemini 2.5 Pro
189
+ - Any model starting with `"gemini-"`
190
+
191
+ **Perplexity Models:**
192
+ - `"online"`, `"sonar"` - Creates `Boxcars::Perplexityai` with Sonar
193
+ - `"sonar-pro"`, `"huge"` - Creates `Boxcars::Perplexityai` with Sonar Pro
194
+ - Models containing `"-sonar-"`
195
+
196
+ **Together AI Models:**
197
+ - `"together-model-name"` - Creates `Boxcars::Together` (strips "together-" prefix)
198
+
199
+ #### JSON-Optimized Engines
200
+
201
+ For applications requiring JSON responses, use the `json_engine` method:
202
+
203
+ ```ruby
204
+ # Creates engine optimized for JSON output
205
+ json_engine = Boxcars::Engines.json_engine(model: "gpt-4o")
206
+
207
+ # Automatically removes response_format for models that don't support it
208
+ json_claude = Boxcars::Engines.json_engine(model: "sonnet")
209
+ ```
210
+
211
+ #### Passing Additional Parameters
212
+
213
+ ```ruby
214
+ # Pass any additional parameters to the underlying engine
215
+ engine = Boxcars::Engines.engine(
216
+ model: "gpt-4o",
217
+ temperature: 0.7,
218
+ max_tokens: 1000,
219
+ top_p: 0.9
220
+ )
221
+ ```
222
+
223
+ #### Using with Boxcars
224
+
225
+ ```ruby
226
+ # Use the factory with any Boxcar
227
+ engine = Boxcars::Engines.engine(model: "sonnet")
228
+ calc = Boxcars::Calculator.new(engine: engine)
229
+ result = calc.run "What is 15 * 23?"
230
+
231
+ # Or in a Train
232
+ boxcars = [
233
+ Boxcars::Calculator.new(engine: Boxcars::Engines.engine(model: "gpt-4o")),
234
+ Boxcars::GoogleSearch.new(engine: Boxcars::Engines.engine(model: "flash"))
235
+ ]
236
+ train = Boxcars.train.new(boxcars: boxcars)
237
+ ```
238
+
239
+ ### Overriding the Default Engine Model
240
+
241
+ Boxcars provides several ways to override the default engine model used throughout your application. The default model is currently `"gemini-2.5-flash-preview-05-20"`, but you can customize this behavior.
242
+
243
+ #### Global Configuration
244
+
245
+ Set a global default model that will be used by `Boxcars::Engines.engine()` when no model is specified:
246
+
247
+ ```ruby
248
+ # Set the default model globally
249
+ Boxcars.configuration.default_model = "gpt-4o"
250
+
251
+ # Now all engines created without specifying a model will use GPT-4o
252
+ engine = Boxcars::Engines.engine # Uses gpt-4o
253
+ calc = Boxcars::Calculator.new # Uses gpt-4o via default engine
254
+ ```
255
+
256
+ #### Configuration Block
257
+
258
+ Use a configuration block for more organized setup:
259
+
260
+ ```ruby
261
+ Boxcars.configure do |config|
262
+ config.default_model = "sonnet" # Use Claude Sonnet as default
263
+ config.logger = Rails.logger # Set custom logger
264
+ config.log_prompts = true # Enable prompt logging
265
+ end
266
+ ```
267
+
268
+ #### Per-Instance Override
269
+
270
+ Override the model for specific engine instances:
271
+
272
+ ```ruby
273
+ # Global default is gemini-flash, but use different models per boxcar
274
+ default_engine = Boxcars::Engines.engine # Uses global default
275
+ gpt_engine = Boxcars::Engines.engine(model: "gpt-4o") # Uses GPT-4o
276
+ claude_engine = Boxcars::Engines.engine(model: "sonnet") # Uses Claude Sonnet
277
+
278
+ # Use different engines for different boxcars
279
+ calc = Boxcars::Calculator.new(engine: gpt_engine)
280
+ search = Boxcars::GoogleSearch.new(engine: claude_engine)
281
+ ```
282
+
283
+ #### Environment-Based Configuration
284
+
285
+ Set the default model via environment variables or initialization:
286
+
287
+ ```ruby
288
+ # In your application initialization (e.g., Rails initializer)
289
+ if Rails.env.production?
290
+ Boxcars.configuration.default_model = "gpt-4o" # Use GPT-4o in production
291
+ elsif Rails.env.development?
292
+ Boxcars.configuration.default_model = "flash" # Use faster Gemini Flash in development
293
+ else
294
+ Boxcars.configuration.default_model = "groq" # Use Groq for testing
295
+ end
296
+ ```
297
+
298
+ #### Model Resolution Priority
299
+
300
+ The `Boxcars::Engines.engine()` method resolves the model in this order:
301
+
302
+ 1. **Explicit model parameter**: `Boxcars::Engines.engine(model: "gpt-4o")`
303
+ 2. **Global configuration**: `Boxcars.configuration.default_model`
304
+ 3. **Built-in default**: `"gemini-2.5-flash-preview-05-20"`
305
+
306
+ #### Supported Model Aliases
307
+
308
+ When setting `default_model`, you can use any of the supported model aliases:
309
+
310
+ ```ruby
311
+ # These are all valid default_model values:
312
+ Boxcars.configuration.default_model = "gpt-4o" # OpenAI GPT-4o
313
+ Boxcars.configuration.default_model = "sonnet" # Claude Sonnet
314
+ Boxcars.configuration.default_model = "flash" # Gemini Flash
315
+ Boxcars.configuration.default_model = "groq" # Groq Llama
316
+ Boxcars.configuration.default_model = "online" # Perplexity Sonar
317
+ ```
318
+
319
+ #### Legacy Engine Configuration
320
+
321
+ You can also override the default engine class (though this is less common):
322
+
323
+ ```ruby
324
+ # Override the default engine class entirely
325
+ Boxcars.configuration.default_engine = Boxcars::Anthropic
326
+
327
+ # Now Boxcars.engine returns Anthropic instead of OpenAI
328
+ default_engine = Boxcars.engine # Returns Boxcars::Anthropic instance
329
+ ```
330
+
331
+ **Note**: When using `default_engine`, the `default_model` setting is ignored since you're specifying the engine class directly.
332
+
333
+ ### Observability
334
+
335
+ Boxcars includes a comprehensive observability system that allows you to track and monitor AI operations across your application. The system provides insights into LLM calls, performance metrics, errors, and usage patterns.
336
+
337
+ #### Core Components
338
+
339
+ **Observability Class**: The central tracking interface that provides a simple `track` method for recording events.
340
+
341
+ **ObservabilityBackend Module**: An interface that defines how tracking backends should be implemented. All backends must include this module and implement a `track` method.
342
+
343
+ **Built-in Backends**:
344
+ - **PosthogBackend**: Sends events to PostHog for analytics and user behavior tracking
345
+ - **MultiBackend**: Allows sending events to multiple backends simultaneously
346
+
347
+ #### Configuration
348
+
349
+ Set up observability by configuring a backend:
350
+
351
+ ```ruby
352
+ # Using PostHog backend
353
+ require 'boxcars/observability_backends/posthog_backend'
354
+
355
+ Boxcars.configure do |config|
356
+ config.observability_backend = Boxcars::PosthogBackend.new(
357
+ api_key: ENV['POSTHOG_API_KEY'] || 'your_posthog_api_key',
358
+ host: 'https://app.posthog.com' # or your self-hosted instance
359
+ )
360
+ end
361
+
362
+ # Using multiple backends
363
+ require 'boxcars/observability_backends/multi_backend'
364
+ backend1 = Boxcars::PosthogBackend.new(api_key: ENV['POSTHOG_API_KEY'])
365
+ backend2 = YourCustomBackend.new
366
+
367
+ Boxcars.configure do |config|
368
+ config.observability_backend = Boxcars::MultiBackend.new([backend1, backend2])
369
+ end
370
+ ```
371
+
372
+ #### Automatic Tracking
373
+
374
+ Boxcars automatically tracks LLM calls with detailed metrics:
375
+
376
+ ```ruby
377
+ # This automatically generates observability events
378
+ engine = Boxcars::Openai.new
379
+ calc = Boxcars::Calculator.new(engine: engine)
380
+ result = calc.run "what is 2 + 2?"
381
+ ```
382
+
383
+ **Tracked Properties Include**:
384
+ - `provider`: The LLM provider (e.g., "openai", "anthropic")
385
+ - `model_name`: The specific model used
386
+ - `prompt_content`: The conversation messages sent to the LLM
387
+ - `inputs`: Any template inputs provided
388
+ - `duration_ms`: Request duration in milliseconds
389
+ - `success`: Whether the call succeeded
390
+ - `status_code`: HTTP response status
391
+ - `error_message`: Error details if the call failed
392
+ - `response_raw_body`: Raw API response
393
+ - `api_call_parameters`: Parameters sent to the API
394
+
395
+ #### Manual Tracking
396
+
397
+ You can also track custom events:
398
+
399
+ ```ruby
400
+ Boxcars::Observability.track(
401
+ event: 'custom_operation',
402
+ properties: {
403
+ user_id: 'user_123',
404
+ operation_type: 'data_processing',
405
+ duration_ms: 150,
406
+ success: true
407
+ }
408
+ )
409
+ ```
410
+
411
+ #### Creating Custom Backends
412
+
413
+ Implement your own backend by including the `ObservabilityBackend` module:
414
+
415
+ ```ruby
416
+ class CustomBackend
417
+ include Boxcars::ObservabilityBackend
418
+
419
+ def track(event:, properties:)
420
+ # Your custom tracking logic here
421
+ puts "Event: #{event}, Properties: #{properties}"
422
+ end
423
+ end
424
+
425
+ Boxcars.configure do |config|
426
+ config.observability_backend = CustomBackend.new
427
+ end
428
+ ```
429
+
430
+ #### Error Handling
431
+
432
+ The observability system is designed to fail silently to prevent tracking issues from disrupting your main application flow. If a backend raises an error, it will be caught and ignored, ensuring your AI operations continue uninterrupted.
433
+
434
+ #### PostHog Integration
435
+
436
+ The PostHog backend requires the `posthog-ruby` gem:
437
+
438
+ ```ruby
439
+ # Add to your Gemfile
440
+ gem 'posthog-ruby'
441
+
442
+ # Configure the backend
443
+ Boxcars.configure do |config|
444
+ config.observability_backend = Boxcars::PosthogBackend.new(
445
+ api_key: ENV['POSTHOG_API_KEY'],
446
+ host: 'https://app.posthog.com',
447
+ on_error: proc { |status, body|
448
+ Rails.logger.warn "PostHog error: #{status} - #{body}"
449
+ }
450
+ )
451
+ end
452
+ ```
453
+
454
+ Events are automatically associated with users when a `user_id` property is provided. Anonymous events use a default identifier.
455
+
456
+
152
457
  ## Development
153
458
 
154
459
  After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.