ruby_llm-infinite_context 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +26 -1
- data/README.md +102 -1
- data/lib/ruby_llm/infinite_context/batch_import.rb +17 -2
- data/lib/ruby_llm/infinite_context/chat_options.rb +82 -2
- data/lib/ruby_llm/infinite_context/compactor.rb +2 -2
- data/lib/ruby_llm/infinite_context/encryption.rb +80 -0
- data/lib/ruby_llm/infinite_context/manager.rb +3 -2
- data/lib/ruby_llm/infinite_context/model.rb +1 -1
- data/lib/ruby_llm/infinite_context/models.rb +3 -0
- data/lib/ruby_llm/infinite_context/pump.rb +14 -5
- data/lib/ruby_llm/infinite_context/version.rb +1 -1
- data/lib/ruby_llm/infinite_context.rb +4 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: dc3d9691bbb8cb8946147e01ba994af1fea69a31385dbd4647b604b27647d78c
|
|
4
|
+
data.tar.gz: 34ffa318976bfd865455586fff35c9134129e6ac3306ae1a5e25be226c367872
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 1b5fbd9b3cd479c683fb394369b66f2b55922485a52f720e2d3cc3c037f330b40626cd5aa00b065c59116ec115be821df4f13420cd6db8086b01f9f16f259263
|
|
7
|
+
data.tar.gz: '08a3a7c55addf87a36384b69639a800d7b9b7a480264e8c8d91977e03989fe9cea5a489b54fd1ebc12a014996e408c742d39485c263953adca514a7ba8bb1f8d'
|
data/CHANGELOG.md
CHANGED
|
@@ -4,7 +4,32 @@ All notable changes to this gem. The format follows
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the gem follows
|
|
5
5
|
[Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
-
## [0.
|
|
7
|
+
## [0.2.0] - not yet released
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- `RubyLLM::InfiniteContext.encrypt_text = true`: Active Record Encryption (non-deterministic)
|
|
12
|
+
on `ric_entries.text` and `ric_nodes.text`. It works if the models load before or after the
|
|
13
|
+
setting, declares `encrypts` once, and cannot be turned off again in the same process. The
|
|
14
|
+
gem never searches these columns in SQL, so every feature works with it on. The README shows
|
|
15
|
+
the setup, the rules, and how to declare it yourself with `to_prepare`.
|
|
16
|
+
- `RubyLLM::InfiniteContext.required_protocol = :chat_completions`: every chat the gem builds
|
|
17
|
+
or uses (the compactor, the pump and `PumpJob`, each turn of `attach` and the
|
|
18
|
+
`Model` concern, and chats passed in by the caller) must use that protocol. Any other
|
|
19
|
+
protocol, none, a chat whose protocol cannot be read, or a fallback to another provider
|
|
20
|
+
raises `RubyLLM::InfiniteContext::ProtocolError` before a request is sent. `BatchImport`
|
|
21
|
+
(`submit` and `collect!`) always raises `ProtocolError` under this rule, before it logs or
|
|
22
|
+
submits anything: provider batch jobs are not chat completions calls. Load old threads with
|
|
23
|
+
`tree.append` and `Pump::Runner` instead.
|
|
24
|
+
- `rake test:encrypted` runs the library tests again with encryption on; `rake` runs both.
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- `Manager#turn` builds the chat before it logs the user's message, so a chat that fails to
|
|
29
|
+
build leaves no half turn in the log.
|
|
30
|
+
- The pump stops and raises on a `ProtocolError` instead of logging it and retrying.
|
|
31
|
+
|
|
32
|
+
## [0.1.0] - 2026-11-09
|
|
8
33
|
|
|
9
34
|
First version. The gem is not on rubygems.org yet.
|
|
10
35
|
|
data/README.md
CHANGED
|
@@ -9,7 +9,9 @@ agent gets tools to zoom into any line, down to the original message.
|
|
|
9
9
|
|
|
10
10
|
## Status
|
|
11
11
|
|
|
12
|
-
0.
|
|
12
|
+
0.2.0 adds Active Record Encryption for the text columns and a way to require one wire
|
|
13
|
+
protocol for every chat the gem uses (see below and `CHANGELOG.md`). The gem has been tested
|
|
14
|
+
end to end on three
|
|
13
15
|
real models: DeepSeek V4.1 Flash (11 of 12 questions right, 33 zooms), Claude Haiku 4.5
|
|
14
16
|
(10 of 12 right, 31 zooms) and GLM-5.3 Flash (10 of 12 right, 26 zooms), each as both
|
|
15
17
|
compactor and reader, on a small synthetic log (14 entries, 12 questions). That is a smoke
|
|
@@ -148,6 +150,103 @@ Compresses an existing long thread at batch prices with `RubyLLM::Batch` (`ask_l
|
|
|
148
150
|
Call `submit` again after `collect!` when the thread is longer than one batch covers:
|
|
149
151
|
later levels become ready as their children finish.
|
|
150
152
|
|
|
153
|
+
A provider batch job is not a chat completions call. When `required_protocol` is set (see
|
|
154
|
+
below), `submit` and `collect!` raise `ProtocolError`; load the thread with `tree.append` and
|
|
155
|
+
`Pump::Runner` instead.
|
|
156
|
+
|
|
157
|
+
### Encrypting the stored text
|
|
158
|
+
|
|
159
|
+
The text columns (`ric_entries.text`, the original messages, and `ric_nodes.text`, the summary
|
|
160
|
+
lines) can use Active Record Encryption. Set up the keys first (`bin/rails
|
|
161
|
+
db:encryption:init`, then put them in your credentials), then turn it on in an initializer:
|
|
162
|
+
|
|
163
|
+
```ruby
|
|
164
|
+
# config/initializers/infinite_context.rb
|
|
165
|
+
RubyLLM::InfiniteContext.encrypt_text = true
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Rules:
|
|
169
|
+
|
|
170
|
+
- **Set it before the app writes a memory row.** It works if the gem's models load before or
|
|
171
|
+
after the initializer. The models are plain classes that Zeitwerk does not reload, so the
|
|
172
|
+
setting holds for the life of the process. You cannot turn it off again in the same process.
|
|
173
|
+
- **Without keys, the first write raises.** Nothing is stored as plain text by mistake.
|
|
174
|
+
- **The gem never searches these columns in SQL.** It reads rows by id and works on the text
|
|
175
|
+
in Ruby, so non-deterministic encryption (the Rails default) is fine. Do not add your own
|
|
176
|
+
`LIKE` or `where(text: ...)` queries on them.
|
|
177
|
+
- **Other columns are not encrypted.** They hold numbers, kinds and the tree key. `ric_views`
|
|
178
|
+
holds only `[level, i]` pairs. Make the tree key an opaque id (a record id, not an email
|
|
179
|
+
address or a phone number).
|
|
180
|
+
- **Column size:** ciphertext is larger than the plain text (base64 and a JSON header, after
|
|
181
|
+
compression). The generator makes `text` columns, which have no practical limit on SQLite
|
|
182
|
+
and Postgres. On MySQL, change them to `mediumtext` or larger.
|
|
183
|
+
- **Rows that exist already** stay plain text and cannot be read once encryption is on. Set
|
|
184
|
+
`config.active_record.encryption.support_unencrypted_data = true`, run
|
|
185
|
+
`RubyLLM::InfiniteContext::Entry.find_each(&:encrypt)` and the same for `Node`, then turn
|
|
186
|
+
that setting off again.
|
|
187
|
+
|
|
188
|
+
If you prefer to declare it yourself, do it once per process, with a guard:
|
|
189
|
+
|
|
190
|
+
```ruby
|
|
191
|
+
# config/application.rb
|
|
192
|
+
config.to_prepare do
|
|
193
|
+
[RubyLLM::InfiniteContext::Entry, RubyLLM::InfiniteContext::Node].each do |model|
|
|
194
|
+
model.encrypts :text unless Array(model.encrypted_attributes).include?(:text)
|
|
195
|
+
end
|
|
196
|
+
end
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
`to_prepare` runs again on every code reload in development, and the gem's models are not
|
|
200
|
+
reloaded. Without the guard, a second `encrypts :text` would wrap the column twice.
|
|
201
|
+
|
|
202
|
+
### Requiring one protocol
|
|
203
|
+
|
|
204
|
+
To make sure no chat the gem uses can reach another API, require the protocol at boot:
|
|
205
|
+
|
|
206
|
+
```ruby
|
|
207
|
+
RubyLLM::InfiniteContext.required_protocol = :chat_completions
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
Then every chat the gem builds or uses must have that protocol: the compactor (the pump and
|
|
211
|
+
`PumpJob`) and each turn of `attach` and the `Model` concern, including chats that you build
|
|
212
|
+
and pass in (for an `acts_as_chat` record, the chat its `to_llm` returns). A chat with another
|
|
213
|
+
protocol, or with none, raises `RubyLLM::InfiniteContext::ProtocolError` before any request
|
|
214
|
+
is sent and before the turn is logged. `BatchImport` always raises `ProtocolError` under
|
|
215
|
+
this rule, before it logs or submits anything, because provider batch jobs are not chat
|
|
216
|
+
completions calls. A chat with fallback models is accepted only when each fallback is a `RubyLLM::Model`
|
|
217
|
+
on the same provider, because RubyLLM drops the protocol when a fallback changes provider.
|
|
218
|
+
|
|
219
|
+
### A locked-down setup
|
|
220
|
+
|
|
221
|
+
For an app that must keep message text private (Fireworks serverless, chat completions only,
|
|
222
|
+
encrypted storage):
|
|
223
|
+
|
|
224
|
+
```ruby
|
|
225
|
+
# config/initializers/infinite_context.rb
|
|
226
|
+
fireworks = RubyLLM.context do |config|
|
|
227
|
+
config.openai_api_key = Rails.application.credentials.fireworks_api_key
|
|
228
|
+
config.openai_api_base = "https://api.fireworks.ai/inference/v1"
|
|
229
|
+
end
|
|
230
|
+
wire = { provider: :openai, protocol: :chat_completions,
|
|
231
|
+
assume_model_exists: true, context: fireworks }
|
|
232
|
+
|
|
233
|
+
RubyLLM::InfiniteContext.required_protocol = :chat_completions
|
|
234
|
+
RubyLLM::InfiniteContext.compactor_chat_options = wire
|
|
235
|
+
RubyLLM::InfiniteContext.encrypt_text = true
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
Pass the same `wire` as `chat_options:` to `attach`, or set `protocol`, `provider` and
|
|
239
|
+
`context` on the `acts_as_chat` record.
|
|
240
|
+
|
|
241
|
+
Load old threads with `append` and the pump, not `BatchImport` (batch jobs are not chat
|
|
242
|
+
completions, so `BatchImport` refuses under this setup):
|
|
243
|
+
|
|
244
|
+
```ruby
|
|
245
|
+
tree = RubyLLM::InfiniteContext::Tree.for("thread-42", compactor: "accounts/fireworks/models/glm-5p3-flash")
|
|
246
|
+
old_messages.each { |m| tree.append(m.role == "assistant" ? :assistant : :user, m.content) }
|
|
247
|
+
RubyLLM::InfiniteContext::Pump::Runner.new(tree).run # plain chat completions, normal prices
|
|
248
|
+
```
|
|
249
|
+
|
|
151
250
|
## How it works
|
|
152
251
|
|
|
153
252
|
- **Log** (`ric_entries`): append-only, never updated or deleted.
|
|
@@ -189,7 +288,9 @@ machinery with no API calls. No client data is used anywhere in the corpus. See
|
|
|
189
288
|
|
|
190
289
|
```
|
|
191
290
|
bin/setup
|
|
291
|
+
rake # the tests, the tests again with encryption on, then standard
|
|
192
292
|
rake test
|
|
293
|
+
rake test:encrypted
|
|
193
294
|
rake standard
|
|
194
295
|
```
|
|
195
296
|
|
|
@@ -31,7 +31,12 @@ module RubyLLM
|
|
|
31
31
|
# Appends `messages` (role/content pairs) to the log and submits the
|
|
32
32
|
# ready nodes as one batch. Returns nil when nothing needs a model
|
|
33
33
|
# call. When RubyLLM::Batch is unavailable, use Pump::Runner instead.
|
|
34
|
+
#
|
|
35
|
+
# With InfiniteContext.required_protocol set, raises ProtocolError
|
|
36
|
+
# before it logs or sends anything: a provider batch job is not a chat
|
|
37
|
+
# completions call.
|
|
34
38
|
def submit(messages)
|
|
39
|
+
refuse_batch_under_required_protocol!
|
|
35
40
|
import_messages(messages)
|
|
36
41
|
chats = []
|
|
37
42
|
nodes = []
|
|
@@ -56,6 +61,7 @@ module RubyLLM
|
|
|
56
61
|
# FIRST: a missing, empty or short result fails the whole collect and
|
|
57
62
|
# stores nothing, so a partial batch never half-populates the tree.
|
|
58
63
|
def collect!(submission, batch)
|
|
64
|
+
refuse_batch_under_required_protocol!
|
|
59
65
|
raise Error, "batch is not complete (#{batch.status})" unless batch.complete?
|
|
60
66
|
|
|
61
67
|
messages = batch.messages
|
|
@@ -80,6 +86,15 @@ module RubyLLM
|
|
|
80
86
|
|
|
81
87
|
private
|
|
82
88
|
|
|
89
|
+
def refuse_batch_under_required_protocol!
|
|
90
|
+
required = InfiniteContext.required_protocol
|
|
91
|
+
return if required.nil?
|
|
92
|
+
|
|
93
|
+
raise ProtocolError, "BatchImport uses provider batch jobs (RubyLLM::Batch), which are not " \
|
|
94
|
+
"#{required.inspect} calls, and RubyLLM::InfiniteContext.required_protocol is #{required.inspect}. " \
|
|
95
|
+
"Load the thread with tree.append for each message, then Pump::Runner.new(tree).run."
|
|
96
|
+
end
|
|
97
|
+
|
|
83
98
|
def import_messages(messages)
|
|
84
99
|
messages.each do |m|
|
|
85
100
|
role = (m[:role] || m["role"] || :user).to_s
|
|
@@ -148,9 +163,9 @@ module RubyLLM
|
|
|
148
163
|
chat = if @chat_factory
|
|
149
164
|
@chat_factory.call
|
|
150
165
|
else
|
|
151
|
-
ChatOptions.build(@model, @chat_options || InfiniteContext.compactor_chat_options)
|
|
166
|
+
ChatOptions.build(@model, @chat_options || InfiniteContext.compactor_chat_options, role: "batch import")
|
|
152
167
|
end
|
|
153
|
-
chat.with_instructions(Prompts::COMPACT_SYSTEM)
|
|
168
|
+
ChatOptions.check!(chat, "batch import").with_instructions(Prompts::COMPACT_SYSTEM)
|
|
154
169
|
end
|
|
155
170
|
|
|
156
171
|
def provider_name(batch)
|
|
@@ -15,6 +15,10 @@ module RubyLLM
|
|
|
15
15
|
# (POST {base}/responses) by default. Fireworks and most other
|
|
16
16
|
# OpenAI-compatible services only offer chat completions, so pass
|
|
17
17
|
# `protocol: :chat_completions` for them. See the README.
|
|
18
|
+
#
|
|
19
|
+
# A host app can REQUIRE one protocol for every chat the gem builds or
|
|
20
|
+
# uses (see InfiniteContext.required_protocol=). Then a chat with any
|
|
21
|
+
# other protocol, or none, raises ProtocolError before a request.
|
|
18
22
|
module ChatOptions
|
|
19
23
|
KEYS = %i[provider protocol assume_model_exists context].freeze
|
|
20
24
|
|
|
@@ -33,10 +37,72 @@ module RubyLLM
|
|
|
33
37
|
end
|
|
34
38
|
|
|
35
39
|
# RubyLLM.chat (or context.chat) with the options passed through.
|
|
36
|
-
|
|
40
|
+
# With a required protocol, the options are checked BEFORE the chat is
|
|
41
|
+
# built and the built chat is checked after.
|
|
42
|
+
def build(model, options = nil, role: "chat")
|
|
37
43
|
options = normalize(options).dup
|
|
44
|
+
check_option!(options[:protocol], role)
|
|
38
45
|
context = options.delete(:context)
|
|
39
|
-
(context || RubyLLM).chat(model: model, **options)
|
|
46
|
+
check!((context || RubyLLM).chat(model: model, **options), role)
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Raises ProtocolError when a protocol is required and `protocol` is
|
|
50
|
+
# not it. Does nothing when no protocol is required.
|
|
51
|
+
def check_option!(protocol, role)
|
|
52
|
+
required = InfiniteContext.required_protocol
|
|
53
|
+
return if required.nil? || protocol.to_s == required.to_s
|
|
54
|
+
|
|
55
|
+
raise ProtocolError, "the #{role} options name protocol #{protocol.inspect}, but " \
|
|
56
|
+
"RubyLLM::InfiniteContext.required_protocol is #{required.inspect}; pass protocol: #{required.inspect}"
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
# Checks a chat the gem is about to use. Returns the chat. When a
|
|
60
|
+
# protocol is required, the chat must use it (none is a failure), and
|
|
61
|
+
# every fallback model must stay on the chat's provider (RubyLLM drops
|
|
62
|
+
# the protocol when a fallback moves to another provider). A chat whose
|
|
63
|
+
# protocol cannot be read fails too.
|
|
64
|
+
#
|
|
65
|
+
# An acts_as_chat record sends through its memoized `to_llm` chat, not
|
|
66
|
+
# through its `protocol` attribute, so for a record the `to_llm` chat
|
|
67
|
+
# is the one checked.
|
|
68
|
+
def check!(chat, role)
|
|
69
|
+
required = InfiniteContext.required_protocol
|
|
70
|
+
return chat if required.nil?
|
|
71
|
+
|
|
72
|
+
sender = chat.respond_to?(:to_llm) ? chat.to_llm : chat
|
|
73
|
+
actual = protocol_of(sender, role)
|
|
74
|
+
unless actual.to_s == required.to_s
|
|
75
|
+
raise ProtocolError, "the #{role} chat uses protocol #{actual.inspect}, but " \
|
|
76
|
+
"RubyLLM::InfiniteContext.required_protocol is #{required.inspect}; build it with protocol: #{required.inspect}"
|
|
77
|
+
end
|
|
78
|
+
check_fallbacks!(sender, role)
|
|
79
|
+
chat
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
def protocol_of(chat, role)
|
|
83
|
+
if defined?(RubyLLM::Chat) && chat.is_a?(RubyLLM::Chat)
|
|
84
|
+
chat.respond_to?(:protocol) ? chat.protocol : chat.instance_variable_get(:@protocol)
|
|
85
|
+
elsif chat.respond_to?(:protocol)
|
|
86
|
+
chat.protocol
|
|
87
|
+
else
|
|
88
|
+
raise ProtocolError, "cannot read the protocol of the #{role} chat (#{chat.class}), " \
|
|
89
|
+
"and RubyLLM::InfiniteContext.required_protocol is set"
|
|
90
|
+
end
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
def check_fallbacks!(chat, role)
|
|
94
|
+
fallbacks = chat.respond_to?(:fallbacks) ? Array(chat.fallbacks) : []
|
|
95
|
+
return if fallbacks.empty?
|
|
96
|
+
|
|
97
|
+
slug = (chat.respond_to?(:provider) && chat.provider.respond_to?(:slug)) ? chat.provider.slug.to_s : nil
|
|
98
|
+
fallbacks.each do |fallback|
|
|
99
|
+
provider = fallback.respond_to?(:provider) ? fallback.provider&.to_s : nil
|
|
100
|
+
next if slug && provider == slug
|
|
101
|
+
|
|
102
|
+
raise ProtocolError, "the #{role} chat has a fallback (#{fallback.respond_to?(:id) ? fallback.id : fallback}) " \
|
|
103
|
+
"that does not name the chat's provider #{slug.inspect}; with required_protocol set, every fallback " \
|
|
104
|
+
"must be a RubyLLM::Model on the same provider"
|
|
105
|
+
end
|
|
40
106
|
end
|
|
41
107
|
end
|
|
42
108
|
|
|
@@ -54,6 +120,20 @@ module RubyLLM
|
|
|
54
120
|
@compactor_chat_options = ChatOptions.normalize(options)
|
|
55
121
|
end
|
|
56
122
|
|
|
123
|
+
# The protocol every chat the gem builds or uses must have, or nil (the
|
|
124
|
+
# default: no rule). Set it once at boot:
|
|
125
|
+
#
|
|
126
|
+
# RubyLLM::InfiniteContext.required_protocol = :chat_completions
|
|
127
|
+
#
|
|
128
|
+
# Then the compactor (pump, PumpJob), BatchImport and every turn of a
|
|
129
|
+
# Manager (attach, the Model concern) raise ProtocolError when their
|
|
130
|
+
# chat has another protocol or none, before any request is sent.
|
|
131
|
+
attr_reader :required_protocol
|
|
132
|
+
|
|
133
|
+
def required_protocol=(protocol)
|
|
134
|
+
@required_protocol = protocol&.to_sym
|
|
135
|
+
end
|
|
136
|
+
|
|
57
137
|
# Attaches a chat to a memory tree; see Manager.attach.
|
|
58
138
|
def attach(...) = Manager.attach(...)
|
|
59
139
|
end
|
|
@@ -55,7 +55,7 @@ module RubyLLM
|
|
|
55
55
|
@model = model
|
|
56
56
|
@chat_options = chat_options && ChatOptions.normalize(chat_options)
|
|
57
57
|
@chat_factory = chat_factory ||
|
|
58
|
-
-> { ChatOptions.build(@model, @chat_options || InfiniteContext.compactor_chat_options) }
|
|
58
|
+
-> { ChatOptions.build(@model, @chat_options || InfiniteContext.compactor_chat_options, role: "compactor") }
|
|
59
59
|
end
|
|
60
60
|
|
|
61
61
|
# Returns [text, tries].
|
|
@@ -89,7 +89,7 @@ module RubyLLM
|
|
|
89
89
|
end
|
|
90
90
|
|
|
91
91
|
def ask(context, step)
|
|
92
|
-
chat = @chat_factory.call
|
|
92
|
+
chat = ChatOptions.check!(@chat_factory.call, "compactor")
|
|
93
93
|
chat = chat.with_instructions(Prompts::COMPACT_SYSTEM) if chat.respond_to?(:with_instructions)
|
|
94
94
|
# The view lines so far are staged WITHOUT a request (ask_later): they
|
|
95
95
|
# ride along with the step in the one request that `ask` makes. A
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module RubyLLM
|
|
4
|
+
module InfiniteContext
|
|
5
|
+
# Active Record Encryption on the text columns of the memory tables:
|
|
6
|
+
# ric_entries.text (the original messages) and ric_nodes.text (the
|
|
7
|
+
# summary lines). The other columns hold numbers, kinds, labels and the
|
|
8
|
+
# tree key, not message text; ric_views.parts holds only [level, i]
|
|
9
|
+
# pairs. Keep the tree key an opaque id (a record id, not a phone number
|
|
10
|
+
# or an email address): it is not encrypted, because the gem finds trees
|
|
11
|
+
# by key.
|
|
12
|
+
#
|
|
13
|
+
# # config/initializers/infinite_context.rb
|
|
14
|
+
# RubyLLM::InfiniteContext.encrypt_text = true
|
|
15
|
+
#
|
|
16
|
+
# The encryption is non-deterministic (the Rails default). The gem never
|
|
17
|
+
# searches or filters on these columns in SQL: it reads them by id and
|
|
18
|
+
# works on the text in Ruby, so nothing breaks.
|
|
19
|
+
#
|
|
20
|
+
# Order rule: set it before the app writes a memory row, in an
|
|
21
|
+
# initializer or at boot. It works whether the models are loaded before
|
|
22
|
+
# or after the setting (the gem's models are plain classes, not reloaded
|
|
23
|
+
# by Zeitwerk). Once on, it stays on for the process: setting it to false
|
|
24
|
+
# after it was true raises Error (even before the models load).
|
|
25
|
+
module Encryption
|
|
26
|
+
COLUMNS = {"Entry" => :text, "Node" => :text}.freeze
|
|
27
|
+
|
|
28
|
+
module_function
|
|
29
|
+
|
|
30
|
+
# Declares `encrypts` on every loaded model that does not have it yet.
|
|
31
|
+
# Safe to call more than once.
|
|
32
|
+
def apply!
|
|
33
|
+
return unless InfiniteContext.encrypt_text?
|
|
34
|
+
|
|
35
|
+
each_model do |model, column|
|
|
36
|
+
next if declared?(model, column)
|
|
37
|
+
|
|
38
|
+
model.encrypts column
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
# True when a loaded gem model encrypts its text column.
|
|
43
|
+
def applied?
|
|
44
|
+
found = false
|
|
45
|
+
each_model { |model, column| found ||= declared?(model, column) }
|
|
46
|
+
found
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def declared?(model, column)
|
|
50
|
+
Array(model.encrypted_attributes).include?(column)
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def each_model
|
|
54
|
+
COLUMNS.each do |name, column|
|
|
55
|
+
next unless InfiniteContext.const_defined?(name, false)
|
|
56
|
+
|
|
57
|
+
model = InfiniteContext.const_get(name, false)
|
|
58
|
+
yield model, column if model.respond_to?(:encrypts)
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
class << self
|
|
64
|
+
def encrypt_text? = @encrypt_text == true
|
|
65
|
+
|
|
66
|
+
# Turns Active Record Encryption on for the gem's text columns. See
|
|
67
|
+
# Encryption for the order rule. Needs Active Record Encryption keys
|
|
68
|
+
# (bin/rails db:encryption:init); without them the first write raises.
|
|
69
|
+
def encrypt_text=(value)
|
|
70
|
+
on = value ? true : false
|
|
71
|
+
if !on && (@encrypt_text == true || Encryption.applied?)
|
|
72
|
+
raise Error, "encrypt_text cannot be turned off once it is on (restart the process)"
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
@encrypt_text = on
|
|
76
|
+
Encryption.apply!
|
|
77
|
+
end
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
end
|
|
@@ -52,11 +52,12 @@ module RubyLLM
|
|
|
52
52
|
end
|
|
53
53
|
|
|
54
54
|
# One turn: log the user's message, ask with the view as instructions,
|
|
55
|
-
# log the reply, pump. A fresh Turn guards and caps the zooms.
|
|
55
|
+
# log the reply, pump. A fresh Turn guards and caps the zooms. The
|
|
56
|
+
# chat is built (and its protocol checked) before anything is logged.
|
|
56
57
|
def turn(text)
|
|
58
|
+
chat = ChatOptions.check!(@chat_factory.call, "turn")
|
|
57
59
|
tree.append(:user, text)
|
|
58
60
|
turn = Turn.new
|
|
59
|
-
chat = @chat_factory.call
|
|
60
61
|
chat.with_instructions("#{Prompts::VIEW_GUIDE}\n\n#{render_view}")
|
|
61
62
|
chat.with_tools(*Tools.for_turn(tree, turn))
|
|
62
63
|
reply = chat.ask(text)
|
|
@@ -57,7 +57,7 @@ module RubyLLM
|
|
|
57
57
|
raise Error, "the record has no model_id yet" if model_id.to_s.empty?
|
|
58
58
|
|
|
59
59
|
record = self
|
|
60
|
-
factory = -> { ChatOptions.build(record.model_id, Model.chat_options_for(record)) }
|
|
60
|
+
factory = -> { ChatOptions.build(record.model_id, Model.chat_options_for(record), role: "record") }
|
|
61
61
|
Manager.attach(factory, key: "#{self.class.name}:#{id}", budget:, tokens:,
|
|
62
62
|
compactor: options[:compactor], compactor_options: options[:compactor_options])
|
|
63
63
|
end
|
|
@@ -5,7 +5,8 @@ module RubyLLM
|
|
|
5
5
|
# The compactor's scheduler. Builds every node that is not built, has its
|
|
6
6
|
# sources, and has its whole context summarized (every view line before
|
|
7
7
|
# the node's end is built). Up to JOBS at once; a failed node is retried
|
|
8
|
-
# up to `retries` times.
|
|
8
|
+
# up to `retries` times. A ProtocolError (see required_protocol) is a
|
|
9
|
+
# setup error: the run stops and raises it.
|
|
9
10
|
module Pump
|
|
10
11
|
# The nodes to start now, in order: levels from 0 up, and in each
|
|
11
12
|
# level from the left. `built` and `busy` are sets of [level, i].
|
|
@@ -116,26 +117,34 @@ module RubyLLM
|
|
|
116
117
|
threads = nodes.map do |node|
|
|
117
118
|
Thread.new do
|
|
118
119
|
ok = false
|
|
120
|
+
fatal = nil
|
|
119
121
|
# In a Rails app each worker thread runs inside the executor (it
|
|
120
122
|
# returns the thread's database connection when it ends).
|
|
121
123
|
Pump.wrap_thread do
|
|
122
124
|
text, tries = @builder.build(*node)
|
|
123
125
|
@tree.store_node!(*node, text, tries:)
|
|
124
126
|
ok = true
|
|
127
|
+
rescue ProtocolError => e
|
|
128
|
+
# A setup error, not a flaky call: no retry, raised below.
|
|
129
|
+
fatal = e
|
|
125
130
|
rescue => e
|
|
126
131
|
log("node #{TreeMath.label(*node)} failed: #{e.class}: #{e.message}")
|
|
127
132
|
end
|
|
128
|
-
done << [node, ok]
|
|
133
|
+
done << [node, ok, fatal]
|
|
129
134
|
end
|
|
130
135
|
end
|
|
131
136
|
# While this thread waits, the workers may autoload constants. With
|
|
132
137
|
# reloading on (Rails development) that needs the load interlock, so
|
|
133
138
|
# the wait must permit concurrent loads or the pass deadlocks.
|
|
134
|
-
Pump.waiting do
|
|
135
|
-
|
|
139
|
+
results = Pump.waiting do
|
|
140
|
+
out = nodes.map { done.pop }
|
|
136
141
|
threads.each(&:join)
|
|
137
|
-
|
|
142
|
+
out
|
|
138
143
|
end
|
|
144
|
+
fatal = results.map(&:last).compact.first
|
|
145
|
+
raise fatal if fatal
|
|
146
|
+
|
|
147
|
+
results.map { |node, ok, _| [node, ok] }
|
|
139
148
|
end
|
|
140
149
|
|
|
141
150
|
def log(message)
|
|
@@ -18,6 +18,9 @@ module RubyLLM
|
|
|
18
18
|
module InfiniteContext
|
|
19
19
|
class Error < StandardError; end
|
|
20
20
|
|
|
21
|
+
# A chat does not use RubyLLM::InfiniteContext.required_protocol.
|
|
22
|
+
class ProtocolError < Error; end
|
|
23
|
+
|
|
21
24
|
NODE = 512 # target size of one summary line
|
|
22
25
|
DEFAULT_VIEW_BUDGET = 128_000 # default consumer view budget, bytes
|
|
23
26
|
COMPACT_VIEW_BUDGET = 24_000 # the compactor's own view, 16-32 KB
|
|
@@ -38,6 +41,7 @@ require_relative "infinite_context/tokens"
|
|
|
38
41
|
require_relative "infinite_context/fold"
|
|
39
42
|
require_relative "infinite_context/prompts"
|
|
40
43
|
require_relative "infinite_context/chat_options"
|
|
44
|
+
require_relative "infinite_context/encryption"
|
|
41
45
|
require_relative "infinite_context/compactor"
|
|
42
46
|
require_relative "infinite_context/pump"
|
|
43
47
|
require_relative "infinite_context/cache_mark"
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ruby_llm-infinite_context
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Christian Genco
|
|
@@ -66,6 +66,7 @@ files:
|
|
|
66
66
|
- lib/ruby_llm/infinite_context/cache_mark.rb
|
|
67
67
|
- lib/ruby_llm/infinite_context/chat_options.rb
|
|
68
68
|
- lib/ruby_llm/infinite_context/compactor.rb
|
|
69
|
+
- lib/ruby_llm/infinite_context/encryption.rb
|
|
69
70
|
- lib/ruby_llm/infinite_context/engine.rb
|
|
70
71
|
- lib/ruby_llm/infinite_context/fold.rb
|
|
71
72
|
- lib/ruby_llm/infinite_context/manager.rb
|