nitro_intelligence 3.1.0 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/docs/README.md +51 -9
- data/lib/nitro_intelligence/assistant.rb +12 -2
- data/lib/nitro_intelligence/assistants.rb +209 -93
- data/lib/nitro_intelligence/client/client.rb +2 -2
- data/lib/nitro_intelligence/client/factory.rb +11 -3
- data/lib/nitro_intelligence/client/handlers/image_handler.rb +3 -1
- data/lib/nitro_intelligence/configuration.rb +2 -0
- data/lib/nitro_intelligence/deprecator.rb +10 -0
- data/lib/nitro_intelligence/pending_review.rb +222 -0
- data/lib/nitro_intelligence/railtie.rb +9 -0
- data/lib/nitro_intelligence/review_decision.rb +34 -0
- data/lib/nitro_intelligence/run_result.rb +129 -0
- data/lib/nitro_intelligence/tool_call_review_interrupt.rb +7 -13
- data/lib/nitro_intelligence/tool_call_review_validator.rb +4 -8
- data/lib/nitro_intelligence/version.rb +1 -1
- data/lib/nitro_intelligence.rb +3 -0
- metadata +6 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 03f41090e337bd4b4ba8f5caf040355d4b8cfca9d81ffcc9b7a8d91eec2ad64e
|
|
4
|
+
data.tar.gz: 0f83490e7d57c8935cdda63579126c3a4d93b22849c1f0f6015948b94ccbb6b8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f908c8f5cfdb7b2b56eb0d4316c1855b6fbdb25ac7355fb0390bbaa42f794e92f44ccc66255c9c8dfcd1b4c552ef8af554f933d3c23f6389a3ec6164b31b6c13
|
|
7
|
+
data.tar.gz: 55b75e6f31b185d3ade2f8939b8ecae69a41a6c54a3df7a1cc8405424b6762e8a788e3e4c4f8680626f4abe638d8027c69fc3e3a4cb2d590b2c196f7faec16f3
|
data/docs/README.md
CHANGED
|
@@ -19,6 +19,8 @@ NitroIntelligence.configure do |config|
|
|
|
19
19
|
# Inference (LLM) settings
|
|
20
20
|
config.inference_api_key = "..." # API key for the inference service
|
|
21
21
|
config.inference_base_url = "https://..." # Base URL for the inference service (optional; defaults to the inference gateway)
|
|
22
|
+
config.inference_max_retries = 0 # Retries the SDK makes itself (optional; defaults to 0 - see "Retries and Timeouts")
|
|
23
|
+
config.inference_timeout = 600 # Request timeout in seconds (optional; defaults to openai-ruby's)
|
|
22
24
|
|
|
23
25
|
# Observability (Langfuse) settings
|
|
24
26
|
config.observability_base_url = "https://..." # Base URL for the observability service (optional; defaults to Cerebro)
|
|
@@ -84,8 +86,11 @@ end
|
|
|
84
86
|
| `cache_provider` | cache store | `NullCache` | ActiveSupport-compatible cache store |
|
|
85
87
|
| `inference_api_key` | `String` | `""` | API key for the LLM inference service |
|
|
86
88
|
| `inference_base_url` | `String` | `"https://inference.powerhome.ai"` | Base URL for the LLM inference service. Defaults to the shared inference gateway, so only a host talking to a different one needs to set it |
|
|
89
|
+
| `inference_max_retries` | `Integer` | `0` | Retries the SDK makes itself after a failed request. Off by default — see [Retries and Timeouts](#retries-and-timeouts) |
|
|
90
|
+
| `inference_timeout` | `Numeric` | `nil` | Request timeout in seconds. `nil` keeps openai-ruby's default of 600 |
|
|
87
91
|
| `observability_base_url` | `String` | `"https://cerebro.powerhome.ai"` | Base URL for the Langfuse observability service. Defaults to Cerebro, so only a host talking to a different one needs to set it |
|
|
88
92
|
| `observability_projects` | `Array<Hash>` | `[]` | Langfuse project credentials (slug, id, public_key, secret_key) |
|
|
93
|
+
| `observability_user_id` | `String` | `""` | User ID recorded on observed traces when a request passes no `user_id` parameter — see [Users](#users) |
|
|
89
94
|
| `assistants_config` | `Hash` | `{}` | Assistants to make addressable by key. `base_url` (String, default: `"https://assistants.powerhome.ai"`) and `user_id` (String, default: `"default-user"`) are shared by every entry; `definitions` (Hash) holds one entry per assistant, keyed by what it is looked up with, each able to override a shared value. Without `definitions` the hash is read as credentials for a single `Assistants.new`, which takes the same two defaults — see [Assistants](#assistants) |
|
|
90
95
|
| `model_config` | `Hash` | `{}` | Model defaults and per-model settings. Top-level keys: `default_text_model`, `default_audio_transcription_model`, `default_image_model`, `default_text_to_speech_model`, and `models` (array of per-model hashes keyed by `name` and `type`, with type-specific options like `aspect_ratios`/`resolutions` for images or `voices`/`response_formats` for TTS) |
|
|
91
96
|
|
|
@@ -121,6 +126,21 @@ client.chat(parameters: { model: "Qwen/Qwen3.8-27B", max_tokens: 1000, temperatu
|
|
|
121
126
|
|
|
122
127
|
For a full list of supported parameters, see the [API reference here](https://developers.openai.com/api/reference/resources/completions/methods/create).
|
|
123
128
|
|
|
129
|
+
#### Retries and Timeouts
|
|
130
|
+
|
|
131
|
+
The client does not retry failed requests by default. openai-ruby on its own retries a `429`, `408`, `409` or `5xx` twice within about two seconds, but none of those is worth repeating that quickly from here: the inference gateway has already failed over between providers before it answers, and a rate limit or exhausted budget it passes on will not clear within a second. Each retry only adds load, and for billable work such as image generation a retry after a timeout can pay twice. A caller that wants retries decides when, typically in a job with backoff, and can tell a rate limit from an exhausted budget first (see the gateway's [error guidance](https://github.com/powerhome/nitro-intelligence/blob/main/docs/inference-gateway.md)).
|
|
132
|
+
|
|
133
|
+
To retry anyway, or to change the timeout, set them for every client in the configuration (`inference_max_retries`, `inference_timeout`), for one client with the same names, or for one request with openai-ruby's `request_options`. The narrower setting wins.
|
|
134
|
+
|
|
135
|
+
```ruby
|
|
136
|
+
# For one client
|
|
137
|
+
client = NitroIntelligence::Client.new(inference_max_retries: 2, inference_timeout: 30)
|
|
138
|
+
|
|
139
|
+
# For one request, with any method
|
|
140
|
+
client.chat(message: "Why is the sky blue?", parameters: { request_options: { max_retries: 2, timeout: 30 } })
|
|
141
|
+
client.generate_image(message: "A bear installing a window", parameters: { request_options: { timeout: 120 } })
|
|
142
|
+
```
|
|
143
|
+
|
|
124
144
|
### Audio Transcription
|
|
125
145
|
|
|
126
146
|
Nitro Intelligence can be used to transcribe audio from a file into text.
|
|
@@ -336,22 +356,38 @@ client.chat(
|
|
|
336
356
|
)
|
|
337
357
|
```
|
|
338
358
|
|
|
339
|
-
### Grouping Traces: Sessions and Tags
|
|
359
|
+
### Grouping Traces: Users, Sessions and Tags
|
|
340
360
|
|
|
341
|
-
`session_id` groups related traces
|
|
342
|
-
them for filtering. Both are optional and are only sent when you set them.
|
|
361
|
+
`user_id` attributes traces to a user of your application, `session_id` groups related traces, and `tags` label them for filtering. All three are optional. `session_id` and `tags` are sent only when you set them.
|
|
343
362
|
|
|
344
363
|
```ruby
|
|
345
364
|
client = NitroIntelligence::Client.new(observability_project_slug: "fake-feature-project")
|
|
346
365
|
client.chat(
|
|
347
366
|
message: "why did this deploy fail?",
|
|
348
367
|
parameters: {
|
|
368
|
+
user_id: current_user.id.to_s,
|
|
349
369
|
session_id: "deploy-9f2c1ab",
|
|
350
370
|
tags: ["deployment-failure-analyzer"],
|
|
351
371
|
}
|
|
352
372
|
)
|
|
353
373
|
```
|
|
354
374
|
|
|
375
|
+
#### Users
|
|
376
|
+
|
|
377
|
+
A trace that carries a `user_id` shows up under that user in Cerebro's [Users](https://langfuse.com/docs/observability/features/users) view, which lists each user with their trace count, token usage and cost, and links through to their traces. The trace list can also be filtered by user.
|
|
378
|
+
|
|
379
|
+
When a request passes no `user_id`, the `observability_user_id` configuration key is used instead, so a host can attribute all of its traces to one identity (the application itself, say) and override it per request for the traces that belong to a person:
|
|
380
|
+
|
|
381
|
+
```ruby
|
|
382
|
+
NitroIntelligence.configure do |config|
|
|
383
|
+
config.observability_user_id = "nitro-web"
|
|
384
|
+
end
|
|
385
|
+
```
|
|
386
|
+
|
|
387
|
+
Use an identifier that is stable and not itself personal data — a database ID rather than an email address or name — since it is stored on every trace and shown in Cerebro. It must be a String of at most 200 characters; langfuse-rb drops any other value with a warning, so convert numeric IDs with `to_s`.
|
|
388
|
+
|
|
389
|
+
`user_id` is recorded only in Cerebro. It is not sent to the inference gateway, so to attribute gateway spend to a user as well, also put the identifier in `metadata` (see below).
|
|
390
|
+
|
|
355
391
|
### Correlating With Application Logs and the Inference Gateway
|
|
356
392
|
|
|
357
393
|
Every observed request is correlated across three systems automatically:
|
|
@@ -369,6 +405,8 @@ Every observed request is correlated across three systems automatically:
|
|
|
369
405
|
* **Inference gateway → observability platform.** When a request fails, the
|
|
370
406
|
gateway's own request identifier is read from the error response and recorded on
|
|
371
407
|
the observation as `litellm_call_id` metadata.
|
|
408
|
+
* **Application users → observability platform.** Pass `user_id` to see a
|
|
409
|
+
user's traces, usage and cost together in Cerebro — see [Users](#users).
|
|
372
410
|
* **Application logs → observability platform.** Put whatever your logs are keyed
|
|
373
411
|
by into `metadata` — it lands on the trace *and* in the gateway's spend logs.
|
|
374
412
|
|
|
@@ -562,7 +600,7 @@ client.chat(
|
|
|
562
600
|
|
|
563
601
|
## Nitro Intelligence Assistants
|
|
564
602
|
|
|
565
|
-
`NitroIntelligence::Assistants` is Nitro Intelligence's lightweight SDK for working with hosted agent threads, runs, and human review flows. It is mainly used to initialize conversation threads, trigger agent runs, inspect
|
|
603
|
+
`NitroIntelligence::Assistants` is Nitro Intelligence's lightweight SDK for working with hosted agent threads, runs, and human review flows. It is mainly used to initialize conversation threads, trigger agent runs, inspect what a paused run is waiting on (tool calls or a reply held for review), and resume paused threads after human reviews.
|
|
566
604
|
|
|
567
605
|
For the full guide, see [ASSISTANTS.md](ASSISTANTS.md). For the service this SDK talks to, see the [Nitro Intelligence Assistants documentation](https://portal.powerapp.cloud/docs/default/system/nip-assistants).
|
|
568
606
|
|
|
@@ -573,14 +611,18 @@ An assistant is addressed by the key it is configured under:
|
|
|
573
611
|
```ruby
|
|
574
612
|
assistant = NitroIntelligence.assistants["candidate-concierge"]
|
|
575
613
|
|
|
576
|
-
assistant.
|
|
614
|
+
result = assistant.run(thread_id: thread.id, messages: messages)
|
|
615
|
+
result.reply # => nil while the reply is held for review
|
|
616
|
+
assistant.review(thread_id: thread.id, reviewer_id: user.email,
|
|
617
|
+
interrupt_id: result.review.interrupt_id, reply: { "action" => "approve" })
|
|
577
618
|
assistant.thread_state(thread_id: thread.id)
|
|
578
619
|
```
|
|
579
620
|
|
|
580
|
-
`
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
621
|
+
`run`, `review`, and the deprecated `await_run` and `review_tool_calls` (see
|
|
622
|
+
[Upgrading to 4.0](ASSISTANTS.md#upgrading-to-40)), supply the assistant's own id, so a
|
|
623
|
+
caller never passes it -- passing `assistant_id:` to any of them raises `ArgumentError` rather
|
|
624
|
+
than quietly sending the call to a different assistant, which is what a migrated caller
|
|
625
|
+
carrying its old argument would otherwise do. Everything else is thread-scoped and delegates to the client untouched. Reach
|
|
584
626
|
the client directly with `assistant.client` if you need it.
|
|
585
627
|
|
|
586
628
|
### Configuration
|
|
@@ -41,20 +41,30 @@ module NitroIntelligence
|
|
|
41
41
|
@client ||= Assistants.new(base_url: @base_url, api_key: @api_key, user_id: @user_id)
|
|
42
42
|
end
|
|
43
43
|
|
|
44
|
-
# The
|
|
44
|
+
# The calls that identify an assistant take it from here rather than from the caller.
|
|
45
45
|
# Everything else is thread-scoped and delegates untouched.
|
|
46
|
+
def run(thread_id:, messages:, **kwargs)
|
|
47
|
+
reject_assistant_id!(kwargs)
|
|
48
|
+
client.run(thread_id:, assistant_id:, messages:, **kwargs)
|
|
49
|
+
end
|
|
50
|
+
|
|
46
51
|
def await_run(thread_id:, messages:, **kwargs)
|
|
47
52
|
reject_assistant_id!(kwargs)
|
|
48
53
|
client.await_run(thread_id:, assistant_id:, messages:, **kwargs)
|
|
49
54
|
end
|
|
50
55
|
|
|
56
|
+
def review(thread_id:, **kwargs)
|
|
57
|
+
reject_assistant_id!(kwargs)
|
|
58
|
+
client.review(thread_id:, assistant_id:, **kwargs)
|
|
59
|
+
end
|
|
60
|
+
|
|
51
61
|
def review_tool_calls(thread_id:, tool_calls:, **kwargs)
|
|
52
62
|
reject_assistant_id!(kwargs)
|
|
53
63
|
client.review_tool_calls(thread_id:, assistant_id:, tool_calls:, **kwargs)
|
|
54
64
|
end
|
|
55
65
|
|
|
56
66
|
delegate :thread_state, :thread_messages, :tool_calls_pending_review, :tool_calls_under_review,
|
|
57
|
-
to: :client
|
|
67
|
+
:under_review, to: :client
|
|
58
68
|
|
|
59
69
|
private
|
|
60
70
|
|
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
require "json"
|
|
2
2
|
require "net/http"
|
|
3
3
|
require "uri"
|
|
4
|
-
require "nitro_intelligence/
|
|
5
|
-
require "nitro_intelligence/
|
|
4
|
+
require "nitro_intelligence/deprecator"
|
|
5
|
+
require "nitro_intelligence/pending_review"
|
|
6
|
+
require "nitro_intelligence/run_result"
|
|
6
7
|
|
|
7
8
|
module NitroIntelligence
|
|
8
9
|
class Assistants
|
|
@@ -11,10 +12,19 @@ module NitroIntelligence
|
|
|
11
12
|
class RunError < StandardError; end
|
|
12
13
|
class ThreadResumptionError < StandardError; end
|
|
13
14
|
class ThreadStateError < StandardError; end
|
|
15
|
+
# The review being answered is no longer the one the thread is paused on. A subclass, so that
|
|
16
|
+
# callers rescuing ThreadResumptionError still catch it.
|
|
17
|
+
class ReviewSupersededError < ThreadResumptionError; end
|
|
18
|
+
|
|
19
|
+
# What Assistants answers, with a 400, to a resume of a thread that is no longer paused.
|
|
20
|
+
NOT_INTERRUPTED_REFUSAL = "thread is not in interrupted state".freeze
|
|
14
21
|
|
|
15
22
|
# Assistants answers with a conflict when `ifExists: "raise"` is sent for a thread that already exists.
|
|
16
23
|
THREAD_CONFLICT_CODE = 409
|
|
17
24
|
|
|
25
|
+
# Where a `/runs/wait` response says which run it waited on.
|
|
26
|
+
RUN_LOCATION = %r{/runs/([^/]+)\z}
|
|
27
|
+
|
|
18
28
|
# Every assistant this gem is built for is served by the same deployment, so a client told
|
|
19
29
|
# nothing about where to reach one gets that deployment. A host talking to a different one
|
|
20
30
|
# -- a review environment, a local server -- still says so.
|
|
@@ -31,11 +41,13 @@ module NitroIntelligence
|
|
|
31
41
|
@base_url = base_url.presence || DEFAULT_BASE_URL
|
|
32
42
|
@api_key = api_key
|
|
33
43
|
@user_id = user_id
|
|
34
|
-
@tool_call_review_validator = ToolCallReviewValidator.new
|
|
35
44
|
@graph_ids = {}
|
|
36
45
|
end
|
|
37
46
|
|
|
38
|
-
|
|
47
|
+
# Runs the last of `messages` on the thread, seeding a new thread with the ones before it, and
|
|
48
|
+
# waits for the run to end or pause. Returns a RunResult: its `reply` is nil whenever the run
|
|
49
|
+
# paused, and its `review` says what the run is waiting on.
|
|
50
|
+
def run(thread_id:, assistant_id:, messages:, context: {})
|
|
39
51
|
raise RunError, "messages cannot be empty" if messages.blank?
|
|
40
52
|
|
|
41
53
|
initial_state = messages[0..-2]
|
|
@@ -45,6 +57,17 @@ module NitroIntelligence
|
|
|
45
57
|
trigger_run(thread_id:, assistant_id:, context:, last_message:)
|
|
46
58
|
end
|
|
47
59
|
|
|
60
|
+
# The run's reply, or nil when it paused for review or ended without one.
|
|
61
|
+
#
|
|
62
|
+
# Deprecated: #run says why there is no reply, and how the platform delivered the one there is.
|
|
63
|
+
def await_run(thread_id:, assistant_id:, messages:, context: {})
|
|
64
|
+
NitroIntelligence.deprecator.warn(
|
|
65
|
+
"#await_run is deprecated and is removed in 5.0; use #run(...).reply, and read #run's " \
|
|
66
|
+
"interrupted? and review for a run paused for review"
|
|
67
|
+
)
|
|
68
|
+
run(thread_id:, assistant_id:, messages:, context:).reply
|
|
69
|
+
end
|
|
70
|
+
|
|
48
71
|
# The thread's state as Assistants reports it, unformatted. Callers that only want the
|
|
49
72
|
# conversation should reach for #thread_messages instead.
|
|
50
73
|
def thread_state(thread_id:)
|
|
@@ -57,7 +80,17 @@ module NitroIntelligence
|
|
|
57
80
|
messages_in(thread_state(thread_id:))
|
|
58
81
|
end
|
|
59
82
|
|
|
83
|
+
# Every tool call in the thread that has no result yet, whether or not it is held for review,
|
|
84
|
+
# with the id of the message before it.
|
|
85
|
+
#
|
|
86
|
+
# Deprecated: a reviewer decides only the calls the interrupt holds, which #under_review reports.
|
|
87
|
+
# nitro-web's VirtualConfirmationAgent::Client still reads this, since the VCA's interrupt is not
|
|
88
|
+
# in the platform's shape; it goes when the VCA moves onto the platform.
|
|
60
89
|
def tool_calls_pending_review(thread_id:)
|
|
90
|
+
NitroIntelligence.deprecator.warn(
|
|
91
|
+
"#tool_calls_pending_review is deprecated and is removed in 5.0; use " \
|
|
92
|
+
"#under_review(thread_id:)&.tool_calls for the tool calls awaiting a reviewer's decision"
|
|
93
|
+
)
|
|
61
94
|
thread_state = get_thread_state(thread_id:)
|
|
62
95
|
messages = messages_in(thread_state)
|
|
63
96
|
reviewed_tool_call_ids = tool_messages(messages).map { |message| message["tool_call_id"] }
|
|
@@ -83,37 +116,95 @@ module NitroIntelligence
|
|
|
83
116
|
# A tool the assistant is not configured to interrupt on runs without review, so an AI message
|
|
84
117
|
# can mix calls under review with calls that are only waiting to be executed. This reports the
|
|
85
118
|
# former; #tool_calls_pending_review reports both.
|
|
119
|
+
#
|
|
120
|
+
# Raises ThreadResumptionError when the state cannot be read, as it always has; #under_review is
|
|
121
|
+
# the plain read.
|
|
122
|
+
#
|
|
123
|
+
# Deprecated in favour of #under_review, which reports a pause on the reply as well.
|
|
86
124
|
def tool_calls_under_review(thread_id:)
|
|
87
|
-
|
|
125
|
+
NitroIntelligence.deprecator.warn(
|
|
126
|
+
"#tool_calls_under_review is deprecated and is removed in 5.0; use " \
|
|
127
|
+
"#under_review(thread_id:)&.tool_calls.to_a"
|
|
128
|
+
)
|
|
129
|
+
PendingReview.from_thread_state(get_thread_state(thread_id:))&.tool_calls || []
|
|
88
130
|
end
|
|
89
131
|
|
|
90
|
-
#
|
|
91
|
-
#
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
132
|
+
# What the thread is paused on -- its tool calls or its reply -- or nil when it is not waiting
|
|
133
|
+
# on a review.
|
|
134
|
+
def under_review(thread_id:)
|
|
135
|
+
PendingReview.from_thread_state(get_thread_state(thread_id:, error: ThreadStateError))
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
# Resumes a paused thread with a reviewer's decision on whatever it is paused on, and returns the
|
|
139
|
+
# RunResult of the resumed run. Pass exactly one of:
|
|
140
|
+
#
|
|
141
|
+
# - `reply:`, a single review of the held draft: `{"action" => "approve"}`,
|
|
142
|
+
# `{"action" => "edit", "content" => "..."}` or `{"action" => "reject", "message" => "reason"}`;
|
|
143
|
+
# - `tool_calls:`, one review per held tool call, as #review_tool_calls takes them.
|
|
95
144
|
#
|
|
96
|
-
#
|
|
97
|
-
#
|
|
98
|
-
|
|
145
|
+
# `reviewer_id` and `reviewed_at` (now, unless given) are sent in the resume run's `config.metadata`,
|
|
146
|
+
# where the platform reads them; it records them and never checks them.
|
|
147
|
+
#
|
|
148
|
+
# Pass the `interrupt_id` the reviewer was shown (`PendingReview#interrupt_id`): a thread no longer
|
|
149
|
+
# paused on it raises ReviewSupersededError instead of applying the decision to whatever the thread
|
|
150
|
+
# waits on now. It is required for a reply, since a new message can replace the draft the
|
|
151
|
+
# reviewer read with another, which an approval without it would decide -- and, with a delivery
|
|
152
|
+
# tool, send unread. For tool calls it is optional. The resume is keyed by the interrupt read
|
|
153
|
+
# here, so it never decides a pause that appeared since; reporting such a resume as
|
|
154
|
+
# ReviewSupersededError is best-effort (see PendingReview#unapplied_resume?), as is reporting one
|
|
155
|
+
# Assistants refuses because another review resumed the thread first.
|
|
156
|
+
def review(thread_id:, assistant_id:, reviewer_id:, reply: nil, tool_calls: nil, interrupt_id: nil, # rubocop:disable Metrics/ParameterLists
|
|
157
|
+
reviewed_at: nil, context: {})
|
|
158
|
+
kind, reviews = review_subject!(reply:, tool_calls:)
|
|
159
|
+
raise ArgumentError, "reviewer_id is required" if reviewer_id.blank?
|
|
160
|
+
|
|
161
|
+
if kind == :reply && interrupt_id.blank?
|
|
162
|
+
raise ArgumentError, "interrupt_id is required to review a reply: pass the interrupt_id of the " \
|
|
163
|
+
"review the reviewer was shown (PendingReview#interrupt_id)"
|
|
164
|
+
end
|
|
165
|
+
|
|
166
|
+
pending_review = pending_review!(thread_id:, kind:, interrupt_id:)
|
|
167
|
+
stamp = reviewer_stamp(reviewer_id, reviewed_at)
|
|
168
|
+
|
|
169
|
+
RunResult.new(resume_review(thread_id:, assistant_id:, pending_review:, reviews:, stamp:, context:))
|
|
170
|
+
rescue ThreadResumptionError => e
|
|
171
|
+
raise unless e.message.include?(NOT_INTERRUPTED_REFUSAL)
|
|
172
|
+
|
|
173
|
+
# Assistants refuses a resume once the thread is no longer paused, which is how the first of
|
|
174
|
+
# two reviews deciding at once wins. The second is superseded.
|
|
175
|
+
superseded!(thread_id, pending_review&.interrupt_id)
|
|
176
|
+
end
|
|
177
|
+
|
|
178
|
+
# Resumes an interrupted thread with one review per tool call the interrupt is holding: a hash
|
|
179
|
+
# keyed by tool call id, or a list of reviews that each carry their tool call's `id`. Each names
|
|
180
|
+
# an `action` -- `approve`, `edit`, `reject` or `respond` -- from the decisions the interrupt
|
|
181
|
+
# allows for that tool. `edit` carries `args`, merged over the arguments the model asked for;
|
|
182
|
+
# `respond` carries the `message` returned to the model as the tool's result; `reject` may carry
|
|
183
|
+
# a `message`, recorded as the reason.
|
|
184
|
+
#
|
|
185
|
+
# A thin wrapper over what #review sends for `tool_calls:`: keyed by the interrupt it read, with
|
|
186
|
+
# `reviewer_id` and `reviewed_at` in `config.metadata` when a reviewer is given, and raising
|
|
187
|
+
# ReviewSupersededError when the resumed run shows the decisions were not applied. Unlike #review,
|
|
188
|
+
# a resume Assistants refuses because the thread is no longer paused is raised as it always was, a
|
|
189
|
+
# ThreadResumptionError carrying the response body, which nitro-web parses. Returns nil.
|
|
190
|
+
#
|
|
191
|
+
# Deprecated in favour of #review, which requires the reviewer and returns the resumed run.
|
|
192
|
+
def review_tool_calls(thread_id:, assistant_id:, tool_calls:, context: {}, reviewer_id: nil, reviewed_at: nil)
|
|
193
|
+
NitroIntelligence.deprecator.warn(
|
|
194
|
+
"#review_tool_calls is deprecated and is removed in 5.0; use " \
|
|
195
|
+
"#review(tool_calls:, reviewer_id:, interrupt_id:), which returns the resumed run's RunResult"
|
|
196
|
+
)
|
|
99
197
|
thread = get_thread(thread_id:)
|
|
100
198
|
raise ThreadResumptionError, "Thread #{thread_id} is not in the interrupted state" unless interrupted?(thread)
|
|
101
199
|
|
|
102
|
-
|
|
103
|
-
tool_calls_under_review = interrupt.tool_calls
|
|
200
|
+
pending_review = PendingReview.from_thread_state(get_thread_state(thread_id:))
|
|
104
201
|
|
|
105
|
-
if
|
|
202
|
+
if pending_review.nil? || pending_review.tool_calls.empty?
|
|
106
203
|
raise ThreadResumptionError, "Thread #{thread_id} has no tool calls awaiting review"
|
|
107
204
|
end
|
|
108
205
|
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
resume_run(
|
|
112
|
-
thread_id:,
|
|
113
|
-
assistant_id:,
|
|
114
|
-
resume: { decisions: interrupt.decisions(tool_calls) },
|
|
115
|
-
context:
|
|
116
|
-
)
|
|
206
|
+
stamp = reviewer_id.present? ? reviewer_stamp(reviewer_id, reviewed_at) : nil
|
|
207
|
+
resume_review(thread_id:, assistant_id:, pending_review:, reviews: tool_calls, stamp:, context:)
|
|
117
208
|
|
|
118
209
|
nil
|
|
119
210
|
end
|
|
@@ -132,7 +223,8 @@ module NitroIntelligence
|
|
|
132
223
|
thread_response = create_thread(thread_id:, assistant_id:)
|
|
133
224
|
|
|
134
225
|
return if thread_already_exists?(thread_response)
|
|
135
|
-
|
|
226
|
+
|
|
227
|
+
check!(thread_response, ThreadInitializationError)
|
|
136
228
|
return if initial_state.blank?
|
|
137
229
|
|
|
138
230
|
seed_new_thread_state(thread_id:, initial_state:)
|
|
@@ -173,11 +265,7 @@ module NitroIntelligence
|
|
|
173
265
|
end
|
|
174
266
|
|
|
175
267
|
def fetch_graph_id(assistant_id)
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
raise ThreadInitializationError, assistant_response.body if assistant_response.code.to_i != 200
|
|
179
|
-
|
|
180
|
-
graph_id = JSON.parse(assistant_response.body)["graph_id"]
|
|
268
|
+
graph_id = parse!(get(path: "/assistants/#{assistant_id}"), ThreadInitializationError)["graph_id"]
|
|
181
269
|
|
|
182
270
|
raise ThreadInitializationError, "Assistant #{assistant_id} has no graph_id" if graph_id.blank?
|
|
183
271
|
|
|
@@ -185,14 +273,8 @@ module NitroIntelligence
|
|
|
185
273
|
end
|
|
186
274
|
|
|
187
275
|
def seed_thread_state(thread_id:, initial_state:)
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
body: { values: { messages: initial_state } }
|
|
191
|
-
)
|
|
192
|
-
|
|
193
|
-
raise ThreadInitializationError, state_response.body if state_response.code.to_i != 200
|
|
194
|
-
|
|
195
|
-
JSON.parse(state_response.body)
|
|
276
|
+
parse!(post(path: "/threads/#{thread_id}/state", body: { values: { messages: initial_state } }),
|
|
277
|
+
ThreadInitializationError)
|
|
196
278
|
end
|
|
197
279
|
|
|
198
280
|
def thread_already_exists?(response)
|
|
@@ -202,39 +284,28 @@ module NitroIntelligence
|
|
|
202
284
|
# The review flows have always raised ThreadResumptionError when a state read fails, and consumers
|
|
203
285
|
# rescue it as such. A plain read resumes nothing, so #thread_state asks for ThreadStateError.
|
|
204
286
|
def get_thread_state(thread_id:, error: ThreadResumptionError)
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
raise error, state_response.body if state_response.code.to_i != 200
|
|
208
|
-
|
|
209
|
-
JSON.parse(state_response.body)
|
|
287
|
+
parse!(get(path: "/threads/#{thread_id}/state"), error)
|
|
210
288
|
end
|
|
211
289
|
|
|
212
290
|
def get_thread(thread_id:)
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
raise ThreadResumptionError, thread_response.body if thread_response.code.to_i != 200
|
|
216
|
-
|
|
217
|
-
JSON.parse(thread_response.body)
|
|
291
|
+
parse!(get(path: "/threads/#{thread_id}"), ThreadResumptionError)
|
|
218
292
|
end
|
|
219
293
|
|
|
220
294
|
def trigger_run(thread_id:, assistant_id:, last_message:, context: {})
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
context:,
|
|
226
|
-
input: {
|
|
227
|
-
messages: [last_message],
|
|
228
|
-
},
|
|
229
|
-
}
|
|
230
|
-
)
|
|
231
|
-
|
|
232
|
-
raise RunError, run_response.body if run_response.code.to_i != 200
|
|
295
|
+
run, = wait_for_run(thread_id:, body: { assistant_id:, context:, input: { messages: [last_message] } },
|
|
296
|
+
error: RunError)
|
|
297
|
+
RunResult.new(run)
|
|
298
|
+
end
|
|
233
299
|
|
|
234
|
-
|
|
235
|
-
|
|
300
|
+
# Waits on a run of the thread, raising `error` when the request fails or the run reports an
|
|
301
|
+
# `__error__`. Returns the run and the id Assistants gave it, which the wait reports in its
|
|
302
|
+
# `Content-Location` (`/threads/{thread_id}/runs/{run_id}`); nil without one.
|
|
303
|
+
def wait_for_run(thread_id:, body:, error:)
|
|
304
|
+
response = post(path: "/threads/#{thread_id}/runs/wait", body:)
|
|
305
|
+
run = parse!(response, error)
|
|
306
|
+
raise_run_error!(run, error)
|
|
236
307
|
|
|
237
|
-
|
|
308
|
+
[run, response["Content-Location"].to_s[RUN_LOCATION, 1]]
|
|
238
309
|
end
|
|
239
310
|
|
|
240
311
|
def raise_run_error!(run, error)
|
|
@@ -246,23 +317,66 @@ module NitroIntelligence
|
|
|
246
317
|
end
|
|
247
318
|
|
|
248
319
|
def resume_run(thread_id:, assistant_id:, resume:, context:)
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
320
|
+
wait_for_run(thread_id:, body: { assistant_id:, command: { resume: }, context: }, error: ThreadResumptionError)
|
|
321
|
+
.first
|
|
322
|
+
end
|
|
323
|
+
|
|
324
|
+
def review_subject!(reply:, tool_calls:)
|
|
325
|
+
subjects = { reply:, tool_calls: }.compact
|
|
326
|
+
return subjects.first if subjects.one?
|
|
327
|
+
|
|
328
|
+
raise ArgumentError, "pass exactly one of reply: or tool_calls:"
|
|
329
|
+
end
|
|
330
|
+
|
|
331
|
+
# The thread has to be paused, on the interrupt the reviewer was shown when they say which, and
|
|
332
|
+
# on the kind of review they are answering. A thread whose run is under way again is not
|
|
333
|
+
# paused, whatever its state still reports.
|
|
334
|
+
def pending_review!(thread_id:, kind:, interrupt_id:)
|
|
335
|
+
pending_review = interrupted?(get_thread(thread_id:)) &&
|
|
336
|
+
PendingReview.from_thread_state(get_thread_state(thread_id:))
|
|
337
|
+
|
|
338
|
+
if !pending_review || (interrupt_id.present? && pending_review.interrupt_id != interrupt_id)
|
|
339
|
+
superseded!(thread_id, interrupt_id)
|
|
340
|
+
end
|
|
341
|
+
|
|
342
|
+
if pending_review.kind != kind
|
|
343
|
+
raise ThreadResumptionError,
|
|
344
|
+
"Thread #{thread_id} is awaiting review of #{PendingReview::KIND_NAMES[pending_review.kind]}, " \
|
|
345
|
+
"not of #{PendingReview::KIND_NAMES[kind]}"
|
|
346
|
+
end
|
|
347
|
+
|
|
348
|
+
pending_review
|
|
349
|
+
end
|
|
350
|
+
|
|
351
|
+
def reviewer_stamp(reviewer_id, reviewed_at)
|
|
352
|
+
reviewed_at = (reviewed_at || Time.now).utc.iso8601 unless reviewed_at.is_a?(String)
|
|
353
|
+
|
|
354
|
+
{ "reviewer_id" => reviewer_id.to_s, "reviewed_at" => reviewed_at }
|
|
355
|
+
end
|
|
356
|
+
|
|
357
|
+
# Sends the decisions for `reviews`, validated against the pause, as the resume keyed by the
|
|
358
|
+
# pause's interrupt. A reviewer `stamp` goes in the run's `config.metadata`, where the platform
|
|
359
|
+
# reads it for replies and tool calls alike (LangChain drops extra keys in a tool decision).
|
|
360
|
+
# Raises ReviewSupersededError when the run shows the decisions were not applied.
|
|
361
|
+
def resume_review(thread_id:, assistant_id:, pending_review:, reviews:, stamp:, context:)
|
|
362
|
+
decisions = pending_review.decisions(reviews)
|
|
259
363
|
|
|
260
|
-
|
|
364
|
+
body = { assistant_id:, command: { resume: pending_review.resume(decisions) } }
|
|
365
|
+
body[:config] = { metadata: stamp } if stamp
|
|
366
|
+
body[:context] = context
|
|
261
367
|
|
|
262
|
-
run =
|
|
263
|
-
|
|
368
|
+
run, run_id = wait_for_run(thread_id:, body:, error: ThreadResumptionError)
|
|
369
|
+
return run unless pending_review.unapplied_resume?(run:, run_id:) { get_thread_state(thread_id:) }
|
|
264
370
|
|
|
265
|
-
run
|
|
371
|
+
superseded!(thread_id, pending_review.interrupt_id, now_paused_on: PendingReview.from_run(run).interrupt_id)
|
|
372
|
+
end
|
|
373
|
+
|
|
374
|
+
def superseded!(thread_id, interrupt_id, now_paused_on: nil)
|
|
375
|
+
message = "Thread #{thread_id} is no longer paused on #{interrupt_id.presence || 'a review'}: " \
|
|
376
|
+
"another review already resumed it, or a new message superseded it"
|
|
377
|
+
message += "; it is now paused on #{now_paused_on}, and the decision was not applied" if now_paused_on
|
|
378
|
+
|
|
379
|
+
raise ReviewSupersededError, message
|
|
266
380
|
end
|
|
267
381
|
|
|
268
382
|
def interrupted?(thread)
|
|
@@ -284,37 +398,39 @@ module NitroIntelligence
|
|
|
284
398
|
end
|
|
285
399
|
|
|
286
400
|
def get(path:)
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
http.use_ssl = uri.scheme == "https"
|
|
401
|
+
request(Net::HTTP::Get, path:)
|
|
402
|
+
end
|
|
290
403
|
|
|
291
|
-
|
|
292
|
-
|
|
404
|
+
def delete(path:)
|
|
405
|
+
request(Net::HTTP::Delete, path:)
|
|
406
|
+
end
|
|
293
407
|
|
|
294
|
-
|
|
408
|
+
def post(path:, body:)
|
|
409
|
+
request(Net::HTTP::Post, path:, body:)
|
|
295
410
|
end
|
|
296
411
|
|
|
297
|
-
def
|
|
412
|
+
def request(verb, path:, body: nil)
|
|
298
413
|
uri = URI("#{base_url}#{path}")
|
|
299
414
|
http = Net::HTTP.new(uri.host, uri.port)
|
|
300
415
|
http.use_ssl = uri.scheme == "https"
|
|
301
416
|
|
|
302
|
-
request =
|
|
417
|
+
request = verb.new(uri)
|
|
303
418
|
request_headers.each { |k, v| request[k] = v }
|
|
419
|
+
request.body = body.to_json unless body.nil?
|
|
304
420
|
|
|
305
421
|
http.request(request)
|
|
306
422
|
end
|
|
307
423
|
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
424
|
+
# Anything but a 200 is a failure, raised as `error` with the response body as its message.
|
|
425
|
+
# Thread creation tells a conflict apart before it gets here.
|
|
426
|
+
def check!(response, error)
|
|
427
|
+
raise error, response.body if response.code.to_i != 200
|
|
312
428
|
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
request.body = body.to_json
|
|
429
|
+
response
|
|
430
|
+
end
|
|
316
431
|
|
|
317
|
-
|
|
432
|
+
def parse!(response, error)
|
|
433
|
+
JSON.parse(check!(response, error).body)
|
|
318
434
|
end
|
|
319
435
|
|
|
320
436
|
def request_headers
|
|
@@ -2,8 +2,8 @@ require "nitro_intelligence/client/factory"
|
|
|
2
2
|
|
|
3
3
|
module NitroIntelligence
|
|
4
4
|
module Client
|
|
5
|
-
def self.new(observability_project_slug: nil)
|
|
6
|
-
Factory.new(observability_project_slug:).build
|
|
5
|
+
def self.new(observability_project_slug: nil, inference_max_retries: nil, inference_timeout: nil)
|
|
6
|
+
Factory.new(observability_project_slug:, inference_max_retries:, inference_timeout:).build
|
|
7
7
|
end
|
|
8
8
|
|
|
9
9
|
def self.validate_model(model)
|
|
@@ -6,8 +6,10 @@ require "nitro_intelligence/observability/project"
|
|
|
6
6
|
module NitroIntelligence
|
|
7
7
|
module Client
|
|
8
8
|
class Factory
|
|
9
|
-
def initialize(observability_project_slug:)
|
|
9
|
+
def initialize(observability_project_slug:, inference_max_retries: nil, inference_timeout: nil)
|
|
10
10
|
@observability_project_slug = observability_project_slug
|
|
11
|
+
@inference_max_retries = inference_max_retries
|
|
12
|
+
@inference_timeout = inference_timeout
|
|
11
13
|
end
|
|
12
14
|
|
|
13
15
|
def build
|
|
@@ -42,10 +44,16 @@ module NitroIntelligence
|
|
|
42
44
|
project_client
|
|
43
45
|
end
|
|
44
46
|
|
|
47
|
+
# The gateway owns failover between providers, and an upstream 429 it passes
|
|
48
|
+
# on has already been decided there, so the SDK's own retries default to off
|
|
49
|
+
# rather than to openai-ruby's two. A nil timeout keeps openai-ruby's default.
|
|
45
50
|
def inference_client
|
|
51
|
+
config = NitroIntelligence.config
|
|
46
52
|
@inference_client ||= OpenAI::Client.new(
|
|
47
|
-
api_key:
|
|
48
|
-
base_url:
|
|
53
|
+
api_key: config.inference_api_key,
|
|
54
|
+
base_url: config.inference_base_url,
|
|
55
|
+
max_retries: @inference_max_retries.nil? ? config.inference_max_retries : @inference_max_retries,
|
|
56
|
+
**{ timeout: @inference_timeout || config.inference_timeout }.compact
|
|
49
57
|
)
|
|
50
58
|
end
|
|
51
59
|
end
|
|
@@ -40,7 +40,9 @@ module NitroIntelligence
|
|
|
40
40
|
},
|
|
41
41
|
},
|
|
42
42
|
}
|
|
43
|
-
|
|
43
|
+
# Deep, so that request options a caller passes (headers, max_retries,
|
|
44
|
+
# timeout) sit alongside the image config rather than replacing it.
|
|
45
|
+
parameters.replace(default_parameters.deep_merge(parameters))
|
|
44
46
|
Client.validate_model(parameters[:model])
|
|
45
47
|
end
|
|
46
48
|
|
|
@@ -13,6 +13,8 @@ module NitroIntelligence
|
|
|
13
13
|
config_accessor :assistants_config, default: {}
|
|
14
14
|
config_accessor :inference_api_key, default: ""
|
|
15
15
|
config_accessor :inference_base_url, default: "https://inference.powerhome.ai"
|
|
16
|
+
config_accessor :inference_max_retries, default: 0
|
|
17
|
+
config_accessor :inference_timeout, default: nil
|
|
16
18
|
config_accessor :model_config, default: {}
|
|
17
19
|
config_accessor :observability_base_url, default: "https://cerebro.powerhome.ai"
|
|
18
20
|
config_accessor :observability_projects, default: []
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
require "active_support/deprecation"
|
|
2
|
+
|
|
3
|
+
module NitroIntelligence
|
|
4
|
+
# Removed in the next major release. A Rails host has it registered with its own deprecators by
|
|
5
|
+
# the Railtie, so its `config.active_support` deprecation settings (log, raise, silence) apply;
|
|
6
|
+
# elsewhere it warns on stderr.
|
|
7
|
+
def self.deprecator
|
|
8
|
+
@deprecator ||= ActiveSupport::Deprecation.new("5.0", "nitro_intelligence")
|
|
9
|
+
end
|
|
10
|
+
end
|
|
@@ -0,0 +1,222 @@
|
|
|
1
|
+
require "active_support/core_ext/hash/indifferent_access"
|
|
2
|
+
require "nitro_intelligence/review_decision"
|
|
3
|
+
require "nitro_intelligence/tool_call_review_interrupt"
|
|
4
|
+
require "nitro_intelligence/tool_call_review_validator"
|
|
5
|
+
|
|
6
|
+
module NitroIntelligence
|
|
7
|
+
# What a paused run is waiting on: the interrupt the thread is holding, and the decisions that
|
|
8
|
+
# answer it.
|
|
9
|
+
#
|
|
10
|
+
# Assistants pauses for two kinds of review, never mixed in one interrupt:
|
|
11
|
+
#
|
|
12
|
+
# - tool calls, held by LangChain's `HumanInTheLoopMiddleware`, one action per call;
|
|
13
|
+
# - the run's final reply, a single action named `__reply__` whose `args` carry the draft's
|
|
14
|
+
# `content`.
|
|
15
|
+
#
|
|
16
|
+
# A turn with tool calls only ever pauses on its tool calls, and the reply is the run's final
|
|
17
|
+
# message, so `#tool_calls` and `#reply` are never both present.
|
|
18
|
+
#
|
|
19
|
+
# Built from a thread state (`GET /threads/{id}/state`, which reports `interrupts`) or from the
|
|
20
|
+
# response of a paused `/runs/wait` (which reports `__interrupt__`); both carry the messages the
|
|
21
|
+
# tool calls are matched back onto.
|
|
22
|
+
class PendingReview
|
|
23
|
+
REPLY_ACTION = "__reply__".freeze
|
|
24
|
+
|
|
25
|
+
# What a reviewer may do with a reply. LangChain's `respond` supplies a tool's result, which a
|
|
26
|
+
# reply does not have, so it is never offered for one, whatever the interrupt lists.
|
|
27
|
+
REPLY_DECISIONS = %w[approve edit reject].freeze
|
|
28
|
+
|
|
29
|
+
KIND_NAMES = { tool_calls: "tool calls", reply: "a reply" }.freeze
|
|
30
|
+
|
|
31
|
+
# The draft reply a reviewer is asked to approve, edit or reject.
|
|
32
|
+
Reply = Struct.new(:content, :allowed_decisions, :description, :interrupt_id, keyword_init: true)
|
|
33
|
+
|
|
34
|
+
# The pending review in a thread state, or nil when the thread is not paused.
|
|
35
|
+
def self.from_thread_state(thread_state)
|
|
36
|
+
interrupt = Array(thread_state["interrupts"]).first
|
|
37
|
+
interrupt && new(interrupt:, messages: Array(thread_state.dig("values", "messages")))
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# The pending review in a `/runs/wait` response, or nil when the run did not pause.
|
|
41
|
+
def self.from_run(run)
|
|
42
|
+
interrupt = Array(run["__interrupt__"]).first
|
|
43
|
+
interrupt && new(interrupt:, messages: Array(run["messages"]))
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def initialize(interrupt:, messages:)
|
|
47
|
+
@interrupt = interrupt.to_h
|
|
48
|
+
@messages = messages
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
# Identifies this pause. A thread that has moved on -- a new message superseded the review, or
|
|
52
|
+
# another reviewer decided it first -- reports a different interrupt, or none.
|
|
53
|
+
def interrupt_id
|
|
54
|
+
@interrupt["id"] || @interrupt["interrupt_id"]
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
# `:tool_calls` or `:reply`.
|
|
58
|
+
def kind
|
|
59
|
+
action_name == REPLY_ACTION ? :reply : :tool_calls
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def tool_calls?
|
|
63
|
+
kind == :tool_calls
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def reply?
|
|
67
|
+
kind == :reply
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
# The tool calls the interrupt is holding, as `Assistants#tool_calls_under_review` reports them.
|
|
71
|
+
# Empty when the pause is on a reply.
|
|
72
|
+
def tool_calls
|
|
73
|
+
return [] unless tool_calls?
|
|
74
|
+
|
|
75
|
+
tool_call_interrupt.tool_calls
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
# The held draft, or nil when the pause is not on a reply.
|
|
79
|
+
def reply
|
|
80
|
+
return nil unless reply?
|
|
81
|
+
|
|
82
|
+
@reply ||= Reply.new(
|
|
83
|
+
content: action_args["content"],
|
|
84
|
+
allowed_decisions:,
|
|
85
|
+
description: action_requests.first["description"],
|
|
86
|
+
interrupt_id:
|
|
87
|
+
)
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
# The `command.resume` that sends `decisions` to this pause, keyed by its interrupt. LangGraph
|
|
91
|
+
# applies a keyed resume to that interrupt only: keyed by one that is no longer pending -- the
|
|
92
|
+
# thread moved on between reading this pause and resuming it -- it decides nothing, and the
|
|
93
|
+
# thread stays paused on the newer interrupt. An interrupt reported without an id can only be
|
|
94
|
+
# resumed unkeyed.
|
|
95
|
+
def resume(decisions)
|
|
96
|
+
interrupt_id.blank? ? { decisions: } : { interrupt_id => { decisions: } }
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# Whether `run`, the response to a keyed resume of this pause, decided nothing because the
|
|
100
|
+
# interrupt was gone by the time the resume arrived. `run_id` is the run Assistants created for
|
|
101
|
+
# the resume, and the block reads the thread's state, only when it is needed.
|
|
102
|
+
#
|
|
103
|
+
# Such a resume comes back paused on the newer interrupt, exactly as a resume that was applied
|
|
104
|
+
# and paused again does -- a turn's tool calls decided, then the run's reply held. What tells
|
|
105
|
+
# them apart is who wrote the thread's latest checkpoint: Assistants records each run's id in
|
|
106
|
+
# the metadata of the checkpoints it writes, and a resume that decided nothing writes none, so
|
|
107
|
+
# the pause it reports still sits on a checkpoint an earlier run wrote.
|
|
108
|
+
#
|
|
109
|
+
# Best-effort, and only ever true on that evidence: without the run's id, the checkpoint's, or a
|
|
110
|
+
# state still on the reported pause (a new message may already have moved it on), or when the
|
|
111
|
+
# state cannot be read, it is false and the result stands as returned. That a stale decision is
|
|
112
|
+
# never applied is guaranteed by the keyed resume, not by this.
|
|
113
|
+
def unapplied_resume?(run:, run_id:)
|
|
114
|
+
paused_on = self.class.from_run(run)&.interrupt_id
|
|
115
|
+
return false if interrupt_id.blank? || run_id.blank? || paused_on.blank? || paused_on == interrupt_id
|
|
116
|
+
|
|
117
|
+
state = yield
|
|
118
|
+
writer = state.dig("metadata", "run_id")
|
|
119
|
+
self.class.from_thread_state(state)&.interrupt_id == paused_on && writer.present? && writer != run_id
|
|
120
|
+
rescue
|
|
121
|
+
false
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
# The `command.resume` decisions for a review of this pause, validated against what the
|
|
125
|
+
# interrupt allows before anything is built, raising `Assistants::ThreadResumptionError`.
|
|
126
|
+
#
|
|
127
|
+
# For a reply, `reviews` is a single review hash. For tool calls, it is either a hash of reviews
|
|
128
|
+
# keyed by tool call id, as `Assistants#review_tool_calls` has always taken them, or a list of
|
|
129
|
+
# reviews that each carry their tool call's `id`.
|
|
130
|
+
def decisions(reviews)
|
|
131
|
+
return [reply_decision(reviews)] if reply?
|
|
132
|
+
|
|
133
|
+
reviews = tool_call_reviews(reviews)
|
|
134
|
+
ToolCallReviewValidator.new.validate!(tool_calls: reviews, tool_calls_under_review: tool_calls)
|
|
135
|
+
tool_call_interrupt.decisions(reviews)
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
private
|
|
139
|
+
|
|
140
|
+
def tool_call_interrupt
|
|
141
|
+
@tool_call_interrupt ||= ToolCallReviewInterrupt.new(
|
|
142
|
+
"interrupts" => [@interrupt],
|
|
143
|
+
"values" => { "messages" => @messages }
|
|
144
|
+
)
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
def interrupt_value
|
|
148
|
+
@interrupt["value"] || {}
|
|
149
|
+
end
|
|
150
|
+
|
|
151
|
+
def action_requests
|
|
152
|
+
Array(interrupt_value["action_requests"])
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
def action_name
|
|
156
|
+
action_requests.first&.dig("name")
|
|
157
|
+
end
|
|
158
|
+
|
|
159
|
+
def action_args
|
|
160
|
+
action_requests.first["args"] || {}
|
|
161
|
+
end
|
|
162
|
+
|
|
163
|
+
def allowed_decisions
|
|
164
|
+
review_config = Array(interrupt_value["review_configs"]).find do |config|
|
|
165
|
+
config["action_name"] == action_name
|
|
166
|
+
end
|
|
167
|
+
|
|
168
|
+
Array(review_config&.dig("allowed_decisions")).map(&:to_s) & REPLY_DECISIONS
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
def tool_call_reviews(reviews)
|
|
172
|
+
return reviews unless reviews.is_a?(Array)
|
|
173
|
+
|
|
174
|
+
reviews.each_with_object({}) do |review, by_id|
|
|
175
|
+
unless review.is_a?(Hash) && review.with_indifferent_access[:id].present?
|
|
176
|
+
raise_error!("Each tool call review in a list must be a hash with the tool call's `id`")
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
review = review.with_indifferent_access
|
|
180
|
+
id = review[:id].to_s
|
|
181
|
+
raise_error!("Tool call `#{id}` is reviewed more than once") if by_id.key?(id)
|
|
182
|
+
|
|
183
|
+
by_id[id] = review.except(:id)
|
|
184
|
+
end
|
|
185
|
+
end
|
|
186
|
+
|
|
187
|
+
# A reply is a single action, so it is reviewed with a single hash.
|
|
188
|
+
def reply_decision(review)
|
|
189
|
+
raise_error!("The reply review must be a hash") unless review.is_a?(Hash)
|
|
190
|
+
|
|
191
|
+
review = review.with_indifferent_access
|
|
192
|
+
action = review[:action].to_s
|
|
193
|
+
|
|
194
|
+
unless allowed_decisions.include?(action)
|
|
195
|
+
raise_error!("Invalid review action `#{action}` for the reply; allowed: #{allowed_decisions.join(', ')}")
|
|
196
|
+
end
|
|
197
|
+
|
|
198
|
+
case action
|
|
199
|
+
when "approve" then ReviewDecision.approve
|
|
200
|
+
when "edit"
|
|
201
|
+
ReviewDecision.edit(REPLY_ACTION, { "content" => text!(review, :content, action:, required: true) })
|
|
202
|
+
when "reject" then ReviewDecision.reject(text!(review, :message, action:, required: false))
|
|
203
|
+
end
|
|
204
|
+
end
|
|
205
|
+
|
|
206
|
+
# A required field must be non-blank text; an optional one may be absent, but not something other
|
|
207
|
+
# than a String.
|
|
208
|
+
def text!(review, field, action:, required:)
|
|
209
|
+
value = review[field]
|
|
210
|
+
case ReviewDecision.message_problem(value, required:)
|
|
211
|
+
when :missing then raise_error!("A `#{action}` review of the reply must include a non-empty `#{field}`")
|
|
212
|
+
when :not_string then raise_error!("`#{field}` for the reply review must be a string")
|
|
213
|
+
end
|
|
214
|
+
|
|
215
|
+
value
|
|
216
|
+
end
|
|
217
|
+
|
|
218
|
+
def raise_error!(message)
|
|
219
|
+
raise Assistants::ThreadResumptionError, message
|
|
220
|
+
end
|
|
221
|
+
end
|
|
222
|
+
end
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
module NitroIntelligence
|
|
2
|
+
# The `command.resume` decisions a review sends, and the check on the `message` a decision
|
|
3
|
+
# carries, shared by tool call reviews and reply reviews. Each caller keeps its own
|
|
4
|
+
# error wording, so this reports what is wrong rather than raising.
|
|
5
|
+
module ReviewDecision
|
|
6
|
+
module_function
|
|
7
|
+
|
|
8
|
+
def approve
|
|
9
|
+
{ "type" => "approve" }
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def edit(name, args)
|
|
13
|
+
{ "type" => "edit", "edited_action" => { "name" => name, "args" => args } }
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
# The middleware falls back to its own wording when a rejection carries no reason.
|
|
17
|
+
def reject(message)
|
|
18
|
+
message.nil? ? { "type" => "reject" } : { "type" => "reject", "message" => message }
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def respond(message)
|
|
22
|
+
{ "type" => "respond", "message" => message }
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
# What is wrong with a decision's `message`, or nil: `:missing` when a required one is absent
|
|
26
|
+
# or blank, `:not_string` when one is given as anything but a String.
|
|
27
|
+
def message_problem(message, required:)
|
|
28
|
+
return (:missing if required) if message.nil?
|
|
29
|
+
return :not_string unless message.is_a?(String)
|
|
30
|
+
|
|
31
|
+
:missing if required && message.blank?
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
require "nitro_intelligence/pending_review"
|
|
2
|
+
|
|
3
|
+
module NitroIntelligence
|
|
4
|
+
# The outcome of one `/runs/wait`: either the run ended, possibly with a reply, or it paused on a
|
|
5
|
+
# review.
|
|
6
|
+
#
|
|
7
|
+
# A run has at most one reply: its final message, an AI message with text and no tool calls. Text
|
|
8
|
+
# the model writes alongside tool calls is working text, never the reply. A paused run's last AI
|
|
9
|
+
# message is a draft awaiting review, so `#reply` is nil whenever the run paused; a reply is final
|
|
10
|
+
# only once a run ends without `__interrupt__`.
|
|
11
|
+
class RunResult
|
|
12
|
+
# How the platform delivered the reply through the assistant's delivery tool, read from the
|
|
13
|
+
# reply's `response_metadata.delivery` stamp. `status` is "delivered" (with `result`, what the
|
|
14
|
+
# tool returned, and `delivered_at`), "failed" (with the tool's `error` and `failed_at`), or
|
|
15
|
+
# "pending": the platform stamps a settled reply "pending" just before it delivers it, so a
|
|
16
|
+
# finished run still showing it was cut off before delivery recorded an outcome. Treat a
|
|
17
|
+
# pending reply as not sent; the next customer message removes it and the model answers both. A
|
|
18
|
+
# field the stamp does not carry, or carries as the wrong type, is nil.
|
|
19
|
+
Delivery = Struct.new(:status, :tool, :attempts, :error, :result, :delivered_at, :failed_at,
|
|
20
|
+
keyword_init: true) do
|
|
21
|
+
def delivered?
|
|
22
|
+
status == "delivered"
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def failed?
|
|
26
|
+
status == "failed"
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
def pending?
|
|
30
|
+
status == "pending"
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# The `response_metadata.review` type the platform stamps on a draft when it pauses on it.
|
|
35
|
+
PENDING_REVIEW = "pending".freeze
|
|
36
|
+
|
|
37
|
+
# The response exactly as Assistants returned it.
|
|
38
|
+
attr_reader :raw
|
|
39
|
+
|
|
40
|
+
def initialize(run)
|
|
41
|
+
@raw = run.to_h
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def interrupted?
|
|
45
|
+
Array(raw["__interrupt__"]).any?
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
# What the paused run is waiting on, or nil when it did not pause.
|
|
49
|
+
def review
|
|
50
|
+
return @review if defined?(@review)
|
|
51
|
+
|
|
52
|
+
@review = PendingReview.from_run(raw)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
# The run's messages as Assistants reports them, oldest first.
|
|
56
|
+
def messages
|
|
57
|
+
Array(raw["messages"])
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
# The text of the run's reply, always a String: content given as blocks is reduced to its text.
|
|
61
|
+
# Nil while the run is paused, and when it ended without one: a rejected draft is removed,
|
|
62
|
+
# leaving a tool result or the human's message last, and an earlier turn's text is never the
|
|
63
|
+
# reply. Blank text is no reply.
|
|
64
|
+
def reply
|
|
65
|
+
reply_message && text(reply_message["content"]).presence
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
# The reply's delivery, or nil when the reply carries no delivery stamp: no reply, or no
|
|
69
|
+
# delivery tool configured for the assistant.
|
|
70
|
+
def delivery
|
|
71
|
+
return @delivery if defined?(@delivery)
|
|
72
|
+
|
|
73
|
+
stamp = reply_message&.dig("response_metadata", "delivery")
|
|
74
|
+
@delivery = stamp.is_a?(Hash) ? build_delivery(stamp) : nil
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
private
|
|
78
|
+
|
|
79
|
+
# The run's last message, when the run ended and that message is the assistant's reply: an AI
|
|
80
|
+
# message with no tool calls. A draft still stamped as awaiting review is never the reply, even
|
|
81
|
+
# in a response that reports no pause.
|
|
82
|
+
def reply_message
|
|
83
|
+
return nil if interrupted?
|
|
84
|
+
|
|
85
|
+
message = messages.last
|
|
86
|
+
return nil unless message.is_a?(Hash)
|
|
87
|
+
return nil if message["type"].present? && message["type"] != "ai"
|
|
88
|
+
return nil if Array(message["tool_calls"]).any?
|
|
89
|
+
return nil if awaiting_review?(message)
|
|
90
|
+
|
|
91
|
+
message
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def awaiting_review?(message)
|
|
95
|
+
review = message.dig("response_metadata", "review")
|
|
96
|
+
review.is_a?(Hash) && review["type"] == PENDING_REVIEW
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# A message's content is a string, or a list of content blocks. Its text is the string, or the
|
|
100
|
+
# bare strings and `text` blocks joined, as the platform reads it (LangChain's `message.text`);
|
|
101
|
+
# other blocks -- tool use, reasoning, images -- are not part of the reply.
|
|
102
|
+
def text(content)
|
|
103
|
+
return content if content.is_a?(String)
|
|
104
|
+
return nil unless content.is_a?(Array)
|
|
105
|
+
|
|
106
|
+
content.filter_map do |block|
|
|
107
|
+
next block if block.is_a?(String)
|
|
108
|
+
|
|
109
|
+
block["text"] if block.is_a?(Hash) && block["type"] == "text" && block["text"].is_a?(String)
|
|
110
|
+
end.join
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
def build_delivery(stamp)
|
|
114
|
+
Delivery.new(
|
|
115
|
+
status: string_or_nil(stamp["status"]),
|
|
116
|
+
tool: string_or_nil(stamp["tool"]),
|
|
117
|
+
attempts: stamp["attempts"].is_a?(Integer) ? stamp["attempts"] : nil,
|
|
118
|
+
error: string_or_nil(stamp["error"]),
|
|
119
|
+
result: stamp["result"],
|
|
120
|
+
delivered_at: string_or_nil(stamp["delivered_at"]),
|
|
121
|
+
failed_at: string_or_nil(stamp["failed_at"])
|
|
122
|
+
)
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
def string_or_nil(value)
|
|
126
|
+
value.is_a?(String) ? value : nil
|
|
127
|
+
end
|
|
128
|
+
end
|
|
129
|
+
end
|
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
require "active_support/core_ext/hash/indifferent_access"
|
|
2
|
+
require "nitro_intelligence/review_decision"
|
|
2
3
|
|
|
3
4
|
module NitroIntelligence
|
|
4
5
|
# The tool calls one interrupt is holding, and the resume payload that answers them.
|
|
@@ -68,14 +69,10 @@ module NitroIntelligence
|
|
|
68
69
|
review = (review || {}).with_indifferent_access
|
|
69
70
|
|
|
70
71
|
case review[:action].to_s
|
|
71
|
-
when "approve"
|
|
72
|
-
|
|
73
|
-
when "
|
|
74
|
-
|
|
75
|
-
when "reject"
|
|
76
|
-
review[:message].nil? ? { "type" => "reject" } : { "type" => "reject", "message" => review[:message] }
|
|
77
|
-
when "respond"
|
|
78
|
-
{ "type" => "respond", "message" => review[:message] }
|
|
72
|
+
when "approve" then ReviewDecision.approve
|
|
73
|
+
when "edit" then ReviewDecision.edit(tool_call["name"], edited_args(tool_call, review))
|
|
74
|
+
when "reject" then ReviewDecision.reject(review[:message])
|
|
75
|
+
when "respond" then ReviewDecision.respond(review[:message])
|
|
79
76
|
else
|
|
80
77
|
raise Assistants::ThreadResumptionError,
|
|
81
78
|
"Tool call #{tool_call['id']} has no review naming `approve`, `edit`, `reject` or `respond`"
|
|
@@ -84,11 +81,8 @@ module NitroIntelligence
|
|
|
84
81
|
|
|
85
82
|
# Edited arguments are merged over the call the model made, so a reviewer changing one of them
|
|
86
83
|
# does not have to restate the rest -- and cannot drop one by omitting it.
|
|
87
|
-
def
|
|
88
|
-
{
|
|
89
|
-
"name" => tool_call["name"],
|
|
90
|
-
"args" => tool_call["args"].merge(review[:args] || {}),
|
|
91
|
-
}
|
|
84
|
+
def edited_args(tool_call, review)
|
|
85
|
+
tool_call["args"].merge(review[:args] || {})
|
|
92
86
|
end
|
|
93
87
|
|
|
94
88
|
def interrupt_value
|
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
require "active_support/core_ext/hash/indifferent_access"
|
|
2
|
+
require "nitro_intelligence/review_decision"
|
|
2
3
|
|
|
3
4
|
module NitroIntelligence
|
|
4
5
|
# The reviews a caller submitted, checked against the tool calls the interrupt is holding, before
|
|
@@ -72,15 +73,10 @@ module NitroIntelligence
|
|
|
72
73
|
end
|
|
73
74
|
|
|
74
75
|
def validate_message!(tool_call_id:, review:, required:)
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
raise_error!("Review for tool call #{tool_call_id} must include a message") if required
|
|
79
|
-
return
|
|
76
|
+
case ReviewDecision.message_problem(review[:message], required:)
|
|
77
|
+
when :missing then raise_error!("Review for tool call #{tool_call_id} must include a message")
|
|
78
|
+
when :not_string then raise_error!("Message for tool call #{tool_call_id} must be a string")
|
|
80
79
|
end
|
|
81
|
-
|
|
82
|
-
raise_error!("Message for tool call #{tool_call_id} must be a string") unless message.is_a?(String)
|
|
83
|
-
raise_error!("Review for tool call #{tool_call_id} must include a message") if required && message.blank?
|
|
84
80
|
end
|
|
85
81
|
|
|
86
82
|
def validate_completeness!(submitted_tool_call_ids:, tool_calls_under_review:)
|
data/lib/nitro_intelligence.rb
CHANGED
|
@@ -11,6 +11,7 @@ require "nitro_intelligence/assistants"
|
|
|
11
11
|
require "nitro_intelligence/client/base"
|
|
12
12
|
require "nitro_intelligence/client/client"
|
|
13
13
|
require "nitro_intelligence/configuration"
|
|
14
|
+
require "nitro_intelligence/deprecator"
|
|
14
15
|
require "nitro_intelligence/media/image_generation"
|
|
15
16
|
require "nitro_intelligence/models/model_catalog"
|
|
16
17
|
require "nitro_intelligence/observability/project_client_registry"
|
|
@@ -48,3 +49,5 @@ module NitroIntelligence
|
|
|
48
49
|
end
|
|
49
50
|
end
|
|
50
51
|
end
|
|
52
|
+
|
|
53
|
+
require "nitro_intelligence/railtie" if defined?(Rails::Railtie)
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: nitro_intelligence
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version:
|
|
4
|
+
version: 4.0.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Igor Artemenko
|
|
@@ -93,6 +93,7 @@ files:
|
|
|
93
93
|
- lib/nitro_intelligence/client/observed.rb
|
|
94
94
|
- lib/nitro_intelligence/client/observers/langfuse_observer.rb
|
|
95
95
|
- lib/nitro_intelligence/configuration.rb
|
|
96
|
+
- lib/nitro_intelligence/deprecator.rb
|
|
96
97
|
- lib/nitro_intelligence/langfuse_extension.rb
|
|
97
98
|
- lib/nitro_intelligence/langfuse_tracer_provider.rb
|
|
98
99
|
- lib/nitro_intelligence/media/audio.rb
|
|
@@ -110,7 +111,11 @@ files:
|
|
|
110
111
|
- lib/nitro_intelligence/observability/prompt_resolver.rb
|
|
111
112
|
- lib/nitro_intelligence/observability/prompt_store.rb
|
|
112
113
|
- lib/nitro_intelligence/observability/upload_handler.rb
|
|
114
|
+
- lib/nitro_intelligence/pending_review.rb
|
|
115
|
+
- lib/nitro_intelligence/railtie.rb
|
|
113
116
|
- lib/nitro_intelligence/reporter.rb
|
|
117
|
+
- lib/nitro_intelligence/review_decision.rb
|
|
118
|
+
- lib/nitro_intelligence/run_result.rb
|
|
114
119
|
- lib/nitro_intelligence/tool_call_review_interrupt.rb
|
|
115
120
|
- lib/nitro_intelligence/tool_call_review_validator.rb
|
|
116
121
|
- lib/nitro_intelligence/trace.rb
|