fast-completions 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +51 -0
- data/README.md +132 -0
- data/lib/fast/completions/cancellation.rb +22 -0
- data/lib/fast/completions/chat.rb +257 -0
- data/lib/fast/completions/client.rb +193 -0
- data/lib/fast/completions/codec.rb +335 -0
- data/lib/fast/completions/dialects.rb +170 -0
- data/lib/fast/completions/errors.rb +149 -0
- data/lib/fast/completions/protocol.rb +476 -0
- data/lib/fast/completions/sse.rb +84 -0
- data/lib/fast/completions/tool.rb +50 -0
- data/lib/fast/completions/types.rb +277 -0
- data/lib/fast/completions/version.rb +7 -0
- data/lib/fast/completions.rb +39 -0
- metadata +74 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: b689b98f53f49ee1e7dcf541b2339f24683d34c6b8331994764ddffc0bdb86df
|
|
4
|
+
data.tar.gz: 815c13df95de3782a8dfd8303d3ad50173fd9c0d5c8795fa8cdd7fc9173df8c6
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 5025050d712a676ac2bb5972506260cc63fa4aaff9ca47d20409e18a9ea756b0096deeea9a5168729d5f3757bf03820a88c680107e718d097946b52b0c3ae9dc
|
|
7
|
+
data.tar.gz: 4715c20690b34fc1d569a9b41ff9f291f3c110d32ed66daa9945989f54b18f53de09b265f6d7c1c11815b0f95201327a3627339968ebe2b0b0f37fbc89468317
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
Notable changes to the fast-completions gem. The format is prose-first,
|
|
4
|
+
one section per released version, newest first.
|
|
5
|
+
|
|
6
|
+
## 0.2.0 (2026-10-15)
|
|
7
|
+
|
|
8
|
+
- Tolerant inbound stream (I61): a malformed SSE chunk no longer kills
|
|
9
|
+
the turn. The parser rejoins a data payload cut across lines by a raw
|
|
10
|
+
newline (the z.ai coalesced-delta malformation observed 2026-10-15),
|
|
11
|
+
the chunk seam repairs raw control characters inside strings (pi
|
|
12
|
+
`parseJsonWithRepair` semantics) and re-parses, and a chunk still
|
|
13
|
+
unparseable after rejoin and repair is dropped and counted
|
|
14
|
+
(`Client#dropped_chunks`) while the stream continues to its natural
|
|
15
|
+
terminal. `StreamError` no longer originates from chunk parsing; the
|
|
16
|
+
missing-finish terminal discipline (§10.5) is unchanged. Healthy
|
|
17
|
+
wires are byte-unaffected — the malformed shapes do not occur on
|
|
18
|
+
them.
|
|
19
|
+
|
|
20
|
+
## 0.1.0
|
|
21
|
+
|
|
22
|
+
- Initial release. `Fast::Completions` is a standalone, fiber-native
|
|
23
|
+
client for the OpenAI-compatible chat-completions wire.
|
|
24
|
+
- The request/stream codec is byte-faithful to pi's wire behavior
|
|
25
|
+
(pi `packages/ai/src/api/openai-completions.ts` is the cited oracle
|
|
26
|
+
throughout the source). Codec arms: the plain OpenAI arm (default),
|
|
27
|
+
the zai arm, and the deepseek arm.
|
|
28
|
+
- Dialects are data: a closed 13-key declaration vocabulary via
|
|
29
|
+
`Fast::Completions.dialect(:name) { ... }`, with the well-known
|
|
30
|
+
`:zai`, `:deepseek`, and `:fireworks` declarations shipped. No model
|
|
31
|
+
catalog, no pricing, no provider registry, no base-URL constants.
|
|
32
|
+
- `Client` holds one persistent `Async::HTTP::Client` and runs inside
|
|
33
|
+
the caller's reactor: TTFB deadline + inactivity read-timeout, task
|
|
34
|
+
cancellation that aborts the request and discards the connection, and
|
|
35
|
+
the frozen error taxonomy (ApiError tree, TimeoutError,
|
|
36
|
+
ConnectionError, CancelledError).
|
|
37
|
+
- `Chat` runs the agent turn: build → stream → yield events → execute
|
|
38
|
+
the tool batch CONCURRENTLY (one reactor task per call, children of
|
|
39
|
+
the step task) with results appended in call order, per-call failure
|
|
40
|
+
isolation (a raised error becomes that call's error result while
|
|
41
|
+
siblings complete), and the batch cancel discipline (a cancel aborts
|
|
42
|
+
the in-flight calls and surfaces `CancelledError` with no partial
|
|
43
|
+
results) — cancellation checkpoints and before/after hooks keyed by
|
|
44
|
+
tool-call id.
|
|
45
|
+
- Replay normalization ports pi's cross-model thinking rules: same-model
|
|
46
|
+
signature replay (`reasoning` / `reasoning_content` / `reasoning_text`),
|
|
47
|
+
`reasoning_details` array replay, cross-model folding to text,
|
|
48
|
+
synthetic results for orphaned tool calls, deepseek's
|
|
49
|
+
`reasoning_content: ""`.
|
|
50
|
+
- Parity harness: the byte fixtures (lane `harness`) pin the wire
|
|
51
|
+
against pi's recorded request bodies.
|
data/README.md
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
# fast-completions
|
|
2
|
+
|
|
3
|
+
`Fast::Completions` is a fast & focused HTTP client for OpenAI-compatible chat
|
|
4
|
+
completions APIs. It speaks `POST {base}/chat/completions` and supports SSE
|
|
5
|
+
streaming. It's implemented on top of [socketry/async-http] and won't block
|
|
6
|
+
your reactor.
|
|
7
|
+
|
|
8
|
+
## Fast & Focused
|
|
9
|
+
|
|
10
|
+
This gem is laser focused on **just the chat completions API**
|
|
11
|
+
|
|
12
|
+
It does:
|
|
13
|
+
|
|
14
|
+
* Streaming chat completions
|
|
15
|
+
* Tool calling ([see the tool docs])
|
|
16
|
+
* Pre-made & Custom "Dialects" (common API variations) ([see the dialect docs])
|
|
17
|
+
|
|
18
|
+
There are also a bunch of things `Fast::Completions` _does not_ do:
|
|
19
|
+
|
|
20
|
+
* No model catalog
|
|
21
|
+
* No other API shapes
|
|
22
|
+
* No universal translation
|
|
23
|
+
* No synchronous requests (streaming only)
|
|
24
|
+
* No images over the wire
|
|
25
|
+
|
|
26
|
+
## Dialects
|
|
27
|
+
|
|
28
|
+
Various providers/models make slight deviations from the "official" API. To
|
|
29
|
+
support a wide variety of shapes, we have a concept of "Dialects." The default
|
|
30
|
+
dialect is `:openai`, and we also ship `:zai`, `:deepseek`, and `:fireworks`.
|
|
31
|
+
|
|
32
|
+
Custom dialects can also be declared at runtime. See the [Dialect Docs] for
|
|
33
|
+
more info.
|
|
34
|
+
|
|
35
|
+
## Installation
|
|
36
|
+
|
|
37
|
+
```ruby
|
|
38
|
+
gem "fast-completions"
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
The only runtime dependency is `async-http`.
|
|
42
|
+
|
|
43
|
+
## Quickstart
|
|
44
|
+
|
|
45
|
+
```ruby
|
|
46
|
+
require "fast/completions"
|
|
47
|
+
|
|
48
|
+
class ReadFile < Fast::Completions::Tool
|
|
49
|
+
description "Read file contents."
|
|
50
|
+
parameters do
|
|
51
|
+
{"type" => "object", "required" => ["path"],
|
|
52
|
+
"properties" => {"path" => {"type" => "string"}}}
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def execute(path:)
|
|
56
|
+
File.read(path)
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
model = Fast::Completions::Model.new(
|
|
61
|
+
id: "my-model",
|
|
62
|
+
reasoning: true,
|
|
63
|
+
context_window: 128_000,
|
|
64
|
+
max_tokens: 4096,
|
|
65
|
+
)
|
|
66
|
+
|
|
67
|
+
client = Fast::Completions::Client.new(
|
|
68
|
+
base_url: "https://api.example.com/v1",
|
|
69
|
+
api_key: ENV["API_KEY"],
|
|
70
|
+
)
|
|
71
|
+
|
|
72
|
+
Async do |task| # fiber-native!
|
|
73
|
+
chat = Fast::Completions::Chat.new(client: client, model: model)
|
|
74
|
+
chat.with_instructions("You are a helpful assistant.")
|
|
75
|
+
.with_tools(ReadFile)
|
|
76
|
+
|
|
77
|
+
answer = chat.step do |event|
|
|
78
|
+
puts event.delta if event.type == :text_delta
|
|
79
|
+
end
|
|
80
|
+
puts answer.to_h.inspect
|
|
81
|
+
end
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Dialects as data
|
|
85
|
+
|
|
86
|
+
A dialect is a declaration. The vocabulary is closed (13 keys); unknown
|
|
87
|
+
keys or values raise `ArgumentError` at declaration time — fail loud,
|
|
88
|
+
never silently mistranslate. `:zai`, `:deepseek`, and `:fireworks` ship
|
|
89
|
+
as declarations:
|
|
90
|
+
|
|
91
|
+
```ruby
|
|
92
|
+
Fast::Completions.dialect(:zai) do
|
|
93
|
+
thinking_format :zai
|
|
94
|
+
zai_tool_stream true
|
|
95
|
+
max_tokens_field "max_tokens"
|
|
96
|
+
supports_developer_role false
|
|
97
|
+
supports_reasoning_effort true
|
|
98
|
+
supports_store false
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
model = Fast::Completions::Model.new(..., dialect: :zai)
|
|
102
|
+
Fast::Completions.dialect(:zai) # lookup (KeyError if undeclared)
|
|
103
|
+
Fast::Completions.dialects # the frozen name→Dialect hash
|
|
104
|
+
Fast::Completions.dialect(:acme) { max_tokens_field "max_tokens" } # declare your own
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Re-declaring a name overrides — last declaration wins.
|
|
108
|
+
|
|
109
|
+
## Error taxonomy
|
|
110
|
+
|
|
111
|
+
The error table moved to [the error docs](docs/reference/errors.md) —
|
|
112
|
+
every class with the status mapping, the timeout and connection families,
|
|
113
|
+
and why `CancelledError` stands outside `Error`.
|
|
114
|
+
|
|
115
|
+
## Parity
|
|
116
|
+
|
|
117
|
+
The request/stream codec is byte-faithful to pi's wire: a fixture
|
|
118
|
+
harness (recorded pi request bodies, SHA256-pinned) drives every codec
|
|
119
|
+
arm through the gem and asserts BYTE equality. See
|
|
120
|
+
`spec/fixtures/wire/` and `wire_parity_spec.rb`.
|
|
121
|
+
|
|
122
|
+
## Fiber-native
|
|
123
|
+
|
|
124
|
+
One persistent `Async::HTTP::Client` per `Fast::Completions::Client`;
|
|
125
|
+
requests run inside the caller's reactor — the gem never creates its
|
|
126
|
+
own. Cancellation (`Async::Cancel` / `Chat#cancel`) aborts the
|
|
127
|
+
in-flight request, discards the connection, and raises
|
|
128
|
+
`CancelledError` with no phantom terminal event.
|
|
129
|
+
[see the tool docs]: docs/reference/tools.md
|
|
130
|
+
[see the dialect docs]: docs/reference/dialects.md
|
|
131
|
+
[Dialect Docs]: docs/reference/dialects.md
|
|
132
|
+
[socketry/async-http]: https://github.com/socketry/async-http
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "async"
|
|
4
|
+
|
|
5
|
+
module Fast
|
|
6
|
+
module Completions
|
|
7
|
+
# Cancellation checkpoints (Specification §11, D8): a no-op outside a
|
|
8
|
+
# reactor; inside one it raises CancelledError when the current
|
|
9
|
+
# Async::Task has been cancelled. Chat calls it before each tool
|
|
10
|
+
# execution and at stream start.
|
|
11
|
+
module Cancellation
|
|
12
|
+
module_function
|
|
13
|
+
|
|
14
|
+
def check
|
|
15
|
+
task = Async::Task.current rescue nil
|
|
16
|
+
return unless task&.cancelled?
|
|
17
|
+
|
|
18
|
+
raise CancelledError, "current task cancelled"
|
|
19
|
+
end
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
@@ -0,0 +1,257 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module Fast
|
|
6
|
+
module Completions
|
|
7
|
+
# The agent turn (Specification §12): build → stream → yield events →
|
|
8
|
+
# on :tool_use execute the tool batch CONCURRENTLY (pi's
|
|
9
|
+
# executeToolCallsParallel shape, tart I52's fiber mechanics) — a
|
|
10
|
+
# serial before-phase, one reactor child per executable call, results
|
|
11
|
+
# appended IN CALL ORDER — and return the final assistant Message.
|
|
12
|
+
# Hooks dispatch IN REGISTRATION ORDER; attribution is BY tool_call
|
|
13
|
+
# id — no Thread/fiber-local pairing anywhere (the I49 discipline).
|
|
14
|
+
class Chat
|
|
15
|
+
def initialize(client:, model:)
|
|
16
|
+
@client = client
|
|
17
|
+
@model = model
|
|
18
|
+
@instructions = nil
|
|
19
|
+
@messages = []
|
|
20
|
+
@tools = []
|
|
21
|
+
@thinking_level = nil
|
|
22
|
+
@max_output_tokens = nil
|
|
23
|
+
@before_tool_hooks = []
|
|
24
|
+
@after_tool_hooks = []
|
|
25
|
+
@cancelled = false
|
|
26
|
+
@tool_batch_pending = false
|
|
27
|
+
@in_flight = nil
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# Chainable history configuration. with_instructions replaces (may
|
|
31
|
+
# be called once — later call replaces, §3).
|
|
32
|
+
def with_instructions(text)
|
|
33
|
+
@instructions = text
|
|
34
|
+
self
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# Additive; Tool classes or instances.
|
|
38
|
+
def with_tools(*tools)
|
|
39
|
+
@tools.concat(tools.map { |tool| tool.is_a?(Class) ? tool.new : tool })
|
|
40
|
+
self
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# The gem never mutates a consumer message (§12).
|
|
44
|
+
def add_message(message)
|
|
45
|
+
@messages << message
|
|
46
|
+
self
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def messages
|
|
50
|
+
@messages.dup
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
# The ladder + nil (nil = off-equivalent: no effort); clamped at
|
|
54
|
+
# request build (§9.3).
|
|
55
|
+
def thinking_level=(level)
|
|
56
|
+
@thinking_level = level
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
def with_thinking_level(level)
|
|
60
|
+
@thinking_level = level
|
|
61
|
+
self
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def with_max_output_tokens(tokens)
|
|
65
|
+
@max_output_tokens = tokens
|
|
66
|
+
self
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
# Multiple blocks register; dispatch in registration order (§12).
|
|
70
|
+
def before_tool_call(&block)
|
|
71
|
+
@before_tool_hooks << block
|
|
72
|
+
self
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
def after_tool_result(&block)
|
|
76
|
+
@after_tool_hooks << block
|
|
77
|
+
self
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
# True when the last turn ended with no pending tool batch (§12).
|
|
81
|
+
def complete?
|
|
82
|
+
!@tool_batch_pending
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
# Cancel the in-flight turn (stream or tool batch): latch +
|
|
86
|
+
# Async::Cancel into the task running #step. The turn aborts,
|
|
87
|
+
# CancelledError surfaces, and NO phantom terminal event is emitted
|
|
88
|
+
# (AC3/D8). Cancelling from the step task ITSELF (inside a HOOK
|
|
89
|
+
# execution — hooks run on the calling fiber) sets the latch only —
|
|
90
|
+
# a self-cancelled async task would swallow the unwinding; the
|
|
91
|
+
# checkpoints below honor the latch instead. A tool execution runs
|
|
92
|
+
# on a batch child, so a cancel from inside a tool task-cancels the
|
|
93
|
+
# step task: async stops the sibling children and the join
|
|
94
|
+
# re-raises (D5).
|
|
95
|
+
def cancel
|
|
96
|
+
@cancelled = true
|
|
97
|
+
current = (Async::Task.current rescue nil)
|
|
98
|
+
@in_flight&.cancel unless current&.equal?(@in_flight)
|
|
99
|
+
nil
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
# One assistant turn + tool execution (§12). Returns the final
|
|
103
|
+
# assistant Message.
|
|
104
|
+
def step(&block)
|
|
105
|
+
raise CancelledError, "chat cancelled" if @cancelled
|
|
106
|
+
Cancellation.check # D8 checkpoint at stream start
|
|
107
|
+
@in_flight = Async::Task.current
|
|
108
|
+
reason = nil
|
|
109
|
+
final = @client.stream(
|
|
110
|
+
model: @model, messages: @messages, instructions: @instructions,
|
|
111
|
+
tools: @tools, thinking: @thinking_level, max_tokens: @max_output_tokens,
|
|
112
|
+
) do |event|
|
|
113
|
+
reason = event.reason if event.type == :done
|
|
114
|
+
block&.call(event)
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
add_message(final) # the Chat's own output (stamped by the accumulator, §12)
|
|
118
|
+
if reason == :tool_use
|
|
119
|
+
@tool_batch_pending = true
|
|
120
|
+
run_tool_batch(final)
|
|
121
|
+
else
|
|
122
|
+
@tool_batch_pending = false
|
|
123
|
+
end
|
|
124
|
+
# @in_flight covers the WHOLE turn — stream AND batch: the batch's
|
|
125
|
+
# children hang off this task, so a cancel landing during the
|
|
126
|
+
# batch (sibling-called or tool-called) reaches them via async's
|
|
127
|
+
# stop-children semantics and re-raises from the join (D5).
|
|
128
|
+
@in_flight = nil
|
|
129
|
+
final
|
|
130
|
+
rescue Async::Cancel
|
|
131
|
+
# A cancel landing between awaits (inside hook execution —
|
|
132
|
+
# async raises Async::Cancel synchronously on self-cancel):
|
|
133
|
+
# surface the gem's CancelledError (D8).
|
|
134
|
+
raise CancelledError, "chat cancelled"
|
|
135
|
+
end
|
|
136
|
+
|
|
137
|
+
private
|
|
138
|
+
|
|
139
|
+
# Two-phase batch executor (§1.1 — pi executeToolCallsParallel's
|
|
140
|
+
# shape). PHASE A, serial on the calling fiber per tool_call block
|
|
141
|
+
# IN CALL ORDER: the D8 checkpoints (latch + Cancellation.check),
|
|
142
|
+
# the before hooks (registration order), then executability
|
|
143
|
+
# resolution — an unknown tool finalizes INLINE with the exact
|
|
144
|
+
# result bytes, NO child task (the immediate arm; before hooks fire
|
|
145
|
+
# for it too — gem continuity over pi's look-up-first). PHASE B:
|
|
146
|
+
# ONE reactor child per executable call (children of the step task
|
|
147
|
+
# — the Sync-inline reactor discipline, tart I52's shape). Results
|
|
148
|
+
# collect INDEX-KEYED (call position) and append IN CALL ORDER
|
|
149
|
+
# after the join — history bytes identical to the serial executor's
|
|
150
|
+
# for every batch (wire-neutral by construction, D7). The join
|
|
151
|
+
# re-raises immediately on cancel — never waits out natural
|
|
152
|
+
# completion — so an aborted batch appends NOTHING and fires NO
|
|
153
|
+
# after hooks (no phantom terminals, D5).
|
|
154
|
+
def run_tool_batch(final)
|
|
155
|
+
results = Array.new(final.content.size)
|
|
156
|
+
executable = [] # ordered specs: [block index, tool_call, tool]
|
|
157
|
+
|
|
158
|
+
final.content.each_with_index do |block, index|
|
|
159
|
+
next unless block.is_a?(Hash) && block[:type] == :tool_call
|
|
160
|
+
|
|
161
|
+
# D8 checkpoint: the async task AND the cancel latch (a cancel
|
|
162
|
+
# from inside a hook cannot task-cancel itself, §12).
|
|
163
|
+
raise CancelledError, "chat cancelled" if @cancelled
|
|
164
|
+
|
|
165
|
+
tool_call = {id: block[:id], name: block[:name], arguments: block[:arguments]}
|
|
166
|
+
Cancellation.check # D8: checkpoint before EACH call's hooks
|
|
167
|
+
@before_tool_hooks.each { |hook| hook.call(tool_call) }
|
|
168
|
+
tool = @tools.find { |candidate| candidate.name == tool_call[:name] }
|
|
169
|
+
unless tool
|
|
170
|
+
# Immediate arm (D3): the unknown-tool outcome, verbatim.
|
|
171
|
+
results[index] = Message.tool_result(
|
|
172
|
+
tool_call_id: tool_call[:id], tool_name: tool_call[:name],
|
|
173
|
+
content: "no such tool: #{tool_call[:name]}", is_error: true)
|
|
174
|
+
next
|
|
175
|
+
end
|
|
176
|
+
executable << [index, tool_call, tool]
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
parent = Async::Task.current
|
|
180
|
+
children = executable.map do |index, tool_call, tool|
|
|
181
|
+
parent.async { run_tool_child(tool_call, tool, index, results) }
|
|
182
|
+
end
|
|
183
|
+
begin
|
|
184
|
+
children.map(&:wait) # the join: re-raises immediately on cancel
|
|
185
|
+
ensure
|
|
186
|
+
# The join surfaces WITHOUT waiting out natural completion — but
|
|
187
|
+
# the children would outlive it: #step's ladder converts the
|
|
188
|
+
# join's Async::Cancel into CancelledError (a StandardError), so
|
|
189
|
+
# async's own stop-children transition never runs and the
|
|
190
|
+
# reactor reparents still-running children. Cancel them HERE:
|
|
191
|
+
# no after hooks, no dangling results for aborted calls (D5 —
|
|
192
|
+
# the serial executor's raise-aborts-the-batch semantics). A
|
|
193
|
+
# no-op for children that already finished.
|
|
194
|
+
children.each(&:cancel)
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
results.each { |result| add_message(result) if result } # CALL ORDER (D7)
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
# ONE batch child (Phase B): the opening D8 checkpoint (a cancel
|
|
201
|
+
# that landed during Phase A or a sibling's execution kills THIS
|
|
202
|
+
# child here — its tool never runs), execute with failure
|
|
203
|
+
# isolation, after hooks INSIDE the child as the call completes
|
|
204
|
+
# (keyed by tool_call id through this closure — never
|
|
205
|
+
# completion-shared state), INDEX-keyed collection.
|
|
206
|
+
def run_tool_child(tool_call, tool, index, results)
|
|
207
|
+
raise CancelledError, "chat cancelled" if @cancelled
|
|
208
|
+
Cancellation.check
|
|
209
|
+
result = execute_tool_result(tool_call, tool)
|
|
210
|
+
@after_tool_hooks.each { |hook| hook.call(tool_call, result) }
|
|
211
|
+
results[index] = result
|
|
212
|
+
end
|
|
213
|
+
|
|
214
|
+
# §11 outcome + D4 failure isolation: a StandardError from the
|
|
215
|
+
# tool's own execution becomes THIS call's error result (siblings
|
|
216
|
+
# unaffected, the join completes — pi executePreparedToolCall);
|
|
217
|
+
# CancelledError re-raises (never a result); Async::Cancel passes
|
|
218
|
+
# through (< Exception, not caught here). An ArgumentError
|
|
219
|
+
# re-raises too: Ruby raises it at the DISPATCH boundary — the
|
|
220
|
+
# tool's execute signature cannot accept the provider's arguments
|
|
221
|
+
# (the tool body never ran) — a tool-declaration defect with no pi
|
|
222
|
+
# analog (JS dispatch ignores arity); today's raise-through
|
|
223
|
+
# behavior is kept for it.
|
|
224
|
+
def execute_tool_result(tool_call, tool)
|
|
225
|
+
Message.tool_result(
|
|
226
|
+
tool_call_id: tool_call[:id], tool_name: tool_call[:name],
|
|
227
|
+
**execute_tool_call(tool_call, tool))
|
|
228
|
+
rescue CancelledError
|
|
229
|
+
raise
|
|
230
|
+
rescue ArgumentError
|
|
231
|
+
raise
|
|
232
|
+
rescue StandardError => e
|
|
233
|
+
Message.tool_result(
|
|
234
|
+
tool_call_id: tool_call[:id], tool_name: tool_call[:name],
|
|
235
|
+
content: e.message.to_s, is_error: true)
|
|
236
|
+
end
|
|
237
|
+
|
|
238
|
+
# §11: String → content; Hash → JSON.generate; {error:} → is_error.
|
|
239
|
+
# The block's arguments stay string-keyed (provider JSON bytes); the
|
|
240
|
+
# execute splash symbolizes without mutating the history block.
|
|
241
|
+
def execute_tool_call(tool_call, tool)
|
|
242
|
+
outcome = tool.execute(**symbolize_keys(tool_call[:arguments]))
|
|
243
|
+
if outcome.is_a?(Hash) && outcome.keys == [:error]
|
|
244
|
+
{content: outcome[:error].to_s, is_error: true}
|
|
245
|
+
elsif outcome.is_a?(Hash)
|
|
246
|
+
{content: JSON.generate(outcome), is_error: false}
|
|
247
|
+
else
|
|
248
|
+
{content: outcome.to_s, is_error: false}
|
|
249
|
+
end
|
|
250
|
+
end
|
|
251
|
+
|
|
252
|
+
def symbolize_keys(hash)
|
|
253
|
+
hash.each_with_object({}) { |(k, v), out| out[k.to_sym] = v }
|
|
254
|
+
end
|
|
255
|
+
end
|
|
256
|
+
end
|
|
257
|
+
end
|
|
@@ -0,0 +1,193 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
require "async"
|
|
5
|
+
require "async/http"
|
|
6
|
+
|
|
7
|
+
module Fast
|
|
8
|
+
module Completions
|
|
9
|
+
# ONE persistent Async::HTTP::Client per instance, fiber-native: every
|
|
10
|
+
# request runs INSIDE the caller's reactor — the client never creates
|
|
11
|
+
# its own (§6).
|
|
12
|
+
class Client
|
|
13
|
+
DEFAULT_TIMEOUT = 30
|
|
14
|
+
|
|
15
|
+
# TTFB deadline + inactivity read-timeout watchdog (D5/OQ-1): one
|
|
16
|
+
# value serves both. A fire cancels the waiting task; `fired` tells
|
|
17
|
+
# the cancel apart from a user's (D8).
|
|
18
|
+
class Watchdog
|
|
19
|
+
def initialize(task, seconds)
|
|
20
|
+
@task = task
|
|
21
|
+
@seconds = seconds
|
|
22
|
+
@timer = nil
|
|
23
|
+
@fired = false
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
attr_reader :fired
|
|
27
|
+
|
|
28
|
+
# (Re)arm: TTFB deadline before the POST, re-armed as the
|
|
29
|
+
# inactivity read-timeout after every body chunk.
|
|
30
|
+
def arm
|
|
31
|
+
stop
|
|
32
|
+
@timer = Async do
|
|
33
|
+
sleep @seconds
|
|
34
|
+
@fired = true
|
|
35
|
+
@task.cancel
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
def stop
|
|
40
|
+
@timer&.stop
|
|
41
|
+
@timer = nil
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
NETWORK_ERRORS = [
|
|
46
|
+
Errno::ECONNREFUSED, Errno::ECONNRESET, Errno::EPIPE, Errno::EHOSTUNREACH,
|
|
47
|
+
Errno::ENETUNREACH, Errno::ETIMEDOUT, EOFError, IOError, SocketError,
|
|
48
|
+
].freeze
|
|
49
|
+
|
|
50
|
+
attr_reader :timeout, :dropped_chunks
|
|
51
|
+
|
|
52
|
+
def initialize(base_url:, api_key:, timeout: DEFAULT_TIMEOUT)
|
|
53
|
+
@endpoint = Async::HTTP::Endpoint.parse(base_url)
|
|
54
|
+
@api_key = api_key
|
|
55
|
+
@timeout = timeout
|
|
56
|
+
@client = Async::HTTP::Client.new(@endpoint)
|
|
57
|
+
@path = @endpoint.path.chomp("/") + "/chat/completions"
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def close
|
|
61
|
+
@client.close
|
|
62
|
+
nil
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
# POST {base}/chat/completions, stream the SSE reply, yield every
|
|
66
|
+
# Event to the block, return the final assistant Message (the
|
|
67
|
+
# accumulator itself — §3). Runs inside the caller's task.
|
|
68
|
+
def stream(model:, messages:, instructions: nil, tools: [], thinking: nil,
|
|
69
|
+
max_tokens: nil, temperature: nil, session_id: nil, &block)
|
|
70
|
+
request = Protocol.build_request(
|
|
71
|
+
model: model, messages: messages, instructions: instructions,
|
|
72
|
+
tools: tools, thinking: thinking, max_tokens: max_tokens,
|
|
73
|
+
temperature: temperature,
|
|
74
|
+
)
|
|
75
|
+
headers = default_headers.merge(session_headers(model.dialect, session_id))
|
|
76
|
+
accumulator = Protocol::StreamAccumulator.new(model, model.dialect)
|
|
77
|
+
|
|
78
|
+
watchdog = Watchdog.new(Async::Task.current, @timeout)
|
|
79
|
+
begin
|
|
80
|
+
watchdog.arm
|
|
81
|
+
response = @client.post(@path, headers: headers, body: JSON.generate(request))
|
|
82
|
+
begin
|
|
83
|
+
raise_api_error(response) unless (200...300).cover?(response.status)
|
|
84
|
+
yield accumulator.start_event if block # :start precedes every block event (§10.6)
|
|
85
|
+
feed_stream(response, accumulator, watchdog, &block)
|
|
86
|
+
ensure
|
|
87
|
+
# Complete bodies return the connection to the pool; an aborted
|
|
88
|
+
# read (cancel/timeout) discards it — a cancelled connection is
|
|
89
|
+
# never reused (AC3/D8).
|
|
90
|
+
begin
|
|
91
|
+
response.finish
|
|
92
|
+
rescue StandardError, Async::Cancel
|
|
93
|
+
nil
|
|
94
|
+
end
|
|
95
|
+
end
|
|
96
|
+
rescue Async::Cancel
|
|
97
|
+
watchdog.fired ? raise(TimeoutError, "no data within #{@timeout}s") : raise(CancelledError, "stream cancelled")
|
|
98
|
+
rescue *NETWORK_ERRORS => e
|
|
99
|
+
raise ConnectionError, "#{e.class}: #{e.message}"
|
|
100
|
+
ensure
|
|
101
|
+
watchdog.stop
|
|
102
|
+
end
|
|
103
|
+
raise TimeoutError, "no data within #{@timeout}s" if watchdog.fired
|
|
104
|
+
|
|
105
|
+
deliver_terminal(accumulator, &block)
|
|
106
|
+
end
|
|
107
|
+
|
|
108
|
+
private
|
|
109
|
+
|
|
110
|
+
def default_headers
|
|
111
|
+
{
|
|
112
|
+
"authorization" => "Bearer #{@api_key}",
|
|
113
|
+
"content-type" => "application/json",
|
|
114
|
+
"accept" => "application/json",
|
|
115
|
+
}
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
# Session-affinity headers per dialect when a session id is supplied
|
|
119
|
+
# (§6; pi openai-completions.ts :776-789, compat-schema.ts :4-9).
|
|
120
|
+
def session_headers(dialect, session_id)
|
|
121
|
+
return {} unless dialect.send_session_affinity_headers && session_id
|
|
122
|
+
|
|
123
|
+
case dialect.session_affinity_format
|
|
124
|
+
when :openrouter
|
|
125
|
+
{"x-session-id" => session_id}
|
|
126
|
+
when :openai_nosession
|
|
127
|
+
{"x-client-request-id" => session_id, "x-session-affinity" => session_id}
|
|
128
|
+
else # :openai
|
|
129
|
+
{"session_id" => session_id, "x-client-request-id" => session_id, "x-session-affinity" => session_id}
|
|
130
|
+
end
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
def feed_stream(response, accumulator, watchdog, &block)
|
|
134
|
+
parser = SSE.new
|
|
135
|
+
@dropped_chunks = 0
|
|
136
|
+
response.body.each do |chunk|
|
|
137
|
+
watchdog.arm # re-arm the inactivity read-timeout per chunk (D5)
|
|
138
|
+
parser.push(chunk).each do |item|
|
|
139
|
+
next if item == SSE::DONE # terminal token; no payload
|
|
140
|
+
|
|
141
|
+
json = parse_chunk(item)
|
|
142
|
+
next if json.nil? # dropped (I61): counted, the stream continues
|
|
143
|
+
|
|
144
|
+
events = accumulator.consume(json)
|
|
145
|
+
events.each { |event| yield event } if block
|
|
146
|
+
end
|
|
147
|
+
end
|
|
148
|
+
end
|
|
149
|
+
|
|
150
|
+
# The chunk seam (I61): parse, on failure ONE repair pass (pi
|
|
151
|
+
# parseJsonWithRepair — raw control characters inside strings
|
|
152
|
+
# escaped) and re-parse. StreamError no longer originates from
|
|
153
|
+
# chunk parsing; §10.5's missing-finish path still covers a dropped
|
|
154
|
+
# finish-carrying chunk.
|
|
155
|
+
def parse_chunk(item)
|
|
156
|
+
JSON.parse(item)
|
|
157
|
+
rescue JSON::ParserError
|
|
158
|
+
parse_repaired(item)
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
# The repair re-parse (its own method: a rescue body's exception
|
|
162
|
+
# needs its own rescue). A payload repair cannot fix is DROPPED —
|
|
163
|
+
# counted, never raised; the stream continues (I61).
|
|
164
|
+
def parse_repaired(item)
|
|
165
|
+
JSON.parse(Protocol.repair_json(item))
|
|
166
|
+
rescue JSON::ParserError
|
|
167
|
+
@dropped_chunks += 1
|
|
168
|
+
nil
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
# Finalize blocks, then the terminal discipline (§10.5): an error
|
|
172
|
+
# stop and a missing finish both emit :error FIRST, then raise
|
|
173
|
+
# StreamError; otherwise :done with the mapped reason. Cancel never
|
|
174
|
+
# reaches here — it unwinds mid-read with NO terminal event (AC3).
|
|
175
|
+
def deliver_terminal(accumulator, &block)
|
|
176
|
+
events, outcome = accumulator.finish
|
|
177
|
+
events.each { |event| yield event } if block
|
|
178
|
+
if outcome[0] == :stream_error
|
|
179
|
+
yield Event.new(type: :error, reason: :error, message: accumulator.partial) if block
|
|
180
|
+
raise StreamError.new(outcome[1], stop_reason: :error)
|
|
181
|
+
end
|
|
182
|
+
yield Event.new(type: :done, reason: outcome[1], message: accumulator.partial) if block
|
|
183
|
+
accumulator.partial
|
|
184
|
+
end
|
|
185
|
+
|
|
186
|
+
# Non-2xx: read the body, map per §8.
|
|
187
|
+
def raise_api_error(response)
|
|
188
|
+
body = response.body.read
|
|
189
|
+
raise Errors.class_for_status(response.status, body).new(status: response.status, body: body)
|
|
190
|
+
end
|
|
191
|
+
end
|
|
192
|
+
end
|
|
193
|
+
end
|