fast-completions 0.5.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4ad51b1c13ffc9236e40151891d9c935967c8ca06b982d43e9871e628900e888
-  data.tar.gz: b814642c835590b0399ff287b0ea2631bf2382766f242ba95a9e8857f9d8a8ae
+  metadata.gz: 9d3db818f96038ddf53481fea90ce309a3a01be84d924e8d4707ce121a9232ba
+  data.tar.gz: a6aa0363ad9ba1c6c2b45e47645137390254d8c1b0c8067e8de7f6cc64ed7e43
 SHA512:
-  metadata.gz: 2bfbd4a381f09305f49fdeb578cbde26cf0599de031efa9af638d88dce48b96c38abbabd52e97071633404cdfa7c52237d05b718389465b25595a333a559b549
-  data.tar.gz: d510754ec3b2da6fb8564d51d1dd6925f7382423fc9312a4618af126c7174679a4641b62f6e537a2c78f4c6e10d68a299ff455c100093a4f9a1857b4ee9cace8
+  metadata.gz: 63a9237cef1bbf316295f96bb6bd13107f5efadef2f0434070070edbd1a74a4668c725d6b7690a73cdfa7d02b750df2c4a85f5a60a1f77b96d0c48d4767c6f20
+  data.tar.gz: 90c846372d3c94697e85de1075d61c7b40525b9c0ad7d2dcc11f632c6204d002e946e5ffb9b62529da0695ebeab930af24413475366ef7d09204f3f233ffefea

data/CHANGELOG.md

@@ -3,6 +3,49 @@
 Notable changes to the fast-completions gem. The format is prose-first,
 one section per released version, newest first.
 
+## 0.6.0 (2026-10-25)
+
+- `Client#raw_stream` — the raw completions surface a proxy needs: the
+  caller's body forwarded verbatim (a String as-is, anything else
+  JSON-generated), every parsed SSE payload yielded to the block in
+  arrival order with the wire's terminal DONE token in its wire
+  position, and `nil` returned — the block is the only output channel.
+  No message accumulation, no tool interpretation, no stop-reason
+  mapping, no salvage, no StreamError, and exactly ONE POST attempt —
+  no auto-retry of any kind; a non-2xx raises the §8 mapping
+  immediately. A malformed payload still rides the I61 chunk seam:
+  one repair pass, then drop-and-count on `dropped_chunks` while the
+  stream continues.
+- The watchdog is now two-deadline: `ttfb_timeout` (status + headers)
+  and `inactivity_timeout` (re-armed after every body chunk), each an
+  optional constructor kwarg and per-call override on `raw_stream`,
+  validated as a positive Numeric, falling back to the shared
+  `timeout` when unset; prefill-scale values (1400 s) are first-class.
+  A fire surfaces the phase-named `TimeoutError` ("no response headers
+  within Ns" / "no data within Ns") — never `CancelledError` — and a
+  user cancel still surfaces `CancelledError`. `Client#stream` keeps
+  its single shared timeout and its byte-identical behavior; the
+  additive default on `Watchdog#arm` is the only existing-signature
+  change.
+- Authentication is optional: `api_key` defaults to nil and the
+  `authorization` header is sent only when the key is present and
+  non-empty — a local engine needs no key. With a key present the
+  header set is unchanged.
+- Docs: a README "Raw streaming" section plus the full reference page
+  (`docs/reference/raw-stream.md`); the README owner-prose gate now
+  rules two zones. Platform unchanged: ruby floor `>= 3.3`, CI matrix
+  3.3/3.4/4.0, no new dependencies.
+- Internal refactor of `Client#raw_stream` (operator review of #7): the
+  method is short orchestration over three private seams — the request
+  seam (payload resolution + the ONE POST under the TTFB arm), the
+  each-event seam (the SSE parser, the per-chunk inactivity re-arm, the
+  DONE yield, the drop-and-count reset), and the cancel→timeout
+  translation — with the timeout phase derived from the response
+  instead of a threading flag. Behavior-identical: signature, yielded
+  events, error classes and messages, no-retry, and the
+  connection-finish discipline are unchanged, and the raw_stream spec
+  passes byte-unmodified.
+
 ## 0.5.0 (2026-10-17)
 
 - Tool-args streaming is now an incremental string-state machine

data/README.md

@@ -126,6 +126,23 @@ requests run inside the caller's reactor — the gem never creates its
 own. Cancellation (`Async::Cancel` / `Chat#cancel`) aborts the
 in-flight request, discards the connection, and raises
 `CancelledError` with no phantom terminal event.
+
+## Raw streaming
+
+`Client#raw_stream` is the low-level surface a proxy or gateway needs:
+the request body goes over the wire verbatim (a Hash is JSON-encoded
+as given, a String is sent untouched — no merging, no filtering, no
+`stream` injection), and the block receives every parsed SSE payload
+in arrival order, with the wire's terminal DONE token yielded in its
+wire position. No message accumulation, no tool interpretation, no
+stop-reason mapping; exactly one POST attempt — nothing is retried.
+The watchdog is two-deadline: `ttfb_timeout` bounds the wait for the
+response headers and `inactivity_timeout` bounds the gap between body
+chunks (re-armed after every chunk) — prefill-scale values such as
+1400 s are first-class, and an unset knob falls back to the shared
+`timeout`. The `authorization` header is sent only when `api_key` is
+present and non-empty, so a local engine needs no key. See
+[the raw-stream reference](docs/reference/raw-stream.md).
 [see the tool docs]: docs/reference/tools.md
 [see the dialect docs]: docs/reference/dialects.md
 [Dialect Docs]: docs/reference/dialects.md

data/lib/fast/completions/client.rb

@@ -34,11 +34,14 @@ module Fast
         # resets with the timer: a retried attempt (I62) must not inherit
         # the previous attempt's fire, or the post-stream guard and the
         # cancel-vs-fired discipline would misread attempt 2 (D5/D8).
-        def arm
+        # The additive seconds parameter (I03) lets one watchdog arm
+        # deadlines of different lengths — raw_stream's split phases —
+        # while a bare arm keeps the constructor value (stream, unchanged).
+        def arm(seconds = @seconds)
           stop
           @fired = false
           @timer = Async do
-            sleep @seconds
+            sleep seconds
             @fired = true
             @task.cancel
           end
@@ -55,15 +58,20 @@ module Fast
         Errno::ENETUNREACH, Errno::ETIMEDOUT, EOFError, IOError, SocketError,
       ].freeze
 
-      attr_reader :timeout, :max_retries, :dropped_chunks
+      attr_reader :timeout, :max_retries, :dropped_chunks, :ttfb_timeout, :inactivity_timeout
 
-      def initialize(base_url:, api_key:, timeout: DEFAULT_TIMEOUT, max_retries: DEFAULT_MAX_RETRIES)
+      def initialize(base_url:, api_key: nil, timeout: DEFAULT_TIMEOUT, max_retries: DEFAULT_MAX_RETRIES,
+        ttfb_timeout: nil, inactivity_timeout: nil)
         raise ArgumentError, "max_retries must be an Integer >= 0" unless max_retries.is_a?(Integer) && max_retries >= 0
+        validate_deadline!(:ttfb_timeout, ttfb_timeout)
+        validate_deadline!(:inactivity_timeout, inactivity_timeout)
 
         @endpoint = Async::HTTP::Endpoint.parse(base_url)
         @api_key = api_key
         @timeout = timeout
         @max_retries = max_retries
+        @ttfb_timeout = ttfb_timeout
+        @inactivity_timeout = inactivity_timeout
         @client = Async::HTTP::Client.new(@endpoint)
         @path = @endpoint.path.chomp("/") + "/chat/completions"
       end
@@ -131,14 +139,93 @@ module Fast
         deliver_terminal(accumulator, &block)
       end
 
+      # The raw surface a proxy needs (I03, BRIEF §3): the caller's
+      # request body forwarded VERBATIM (a String as-is, anything else
+      # JSON.generate'd — no merging, no filtering, no stream
+      # injection, no Model/dialect involvement, no session-affinity
+      # headers), every parsed SSE data payload yielded to the block in
+      # arrival order — the SSE::DONE sentinel in its wire position —
+      # and nil returned: the block is the only output channel. No
+      # Message accumulation, no tool interpretation, no stop-reason
+      # mapping, no salvage, no StreamError — a stream that ends without
+      # the terminal token simply ends. Exactly ONE POST attempt: no
+      # create_response, no retry logic, no x-should-retry consultation;
+      # a non-2xx raises the existing §8 mapping (raise_api_error).
+      #
+      # Watchdog, split by I03: two deadlines. TTFB (status + headers)
+      # arms before the POST; from response receipt on the inactivity
+      # deadline re-arms after every body chunk (D5). Each resolves per
+      # phase — call kwarg > client kwarg > the shared timeout. A fire
+      # cancels the waiting task and surfaces the phase-named
+      # TimeoutError, never CancelledError (D8); a user cancel surfaces
+      # CancelledError. Runs inside the caller's task (§6).
+      #
+      # I05 revision (operator review of #7 — "the raw_stream method
+      # is way too long and not idiomatic ruby"): the method is now
+      # short orchestration over three private seams — the request seam
+      # (post_raw_body), the each-event seam (each_raw_event), and the
+      # cancel translation (raise_raw_cancel). The phase a timeout
+      # fires in is derived from the response itself — headers are
+      # outstanding exactly until the POST returns — so no phase flag
+      # threads through the ladder.
+      def raw_stream(body, ttfb_timeout: nil, inactivity_timeout: nil, &block)
+        ttfb = resolve_deadline(:ttfb_timeout, ttfb_timeout, @ttfb_timeout)
+        idle = resolve_deadline(:inactivity_timeout, inactivity_timeout, @inactivity_timeout)
+
+        watchdog = Watchdog.new(Async::Task.current, ttfb)
+        response = nil
+        begin
+          response = post_raw_body(body, watchdog)
+          raise_api_error(response) unless (200...300).cover?(response.status)
+
+          each_raw_event(response, idle, watchdog, &block)
+        rescue Async::Cancel
+          raise_raw_cancel(watchdog, response, ttfb, idle)
+        rescue *NETWORK_ERRORS => e
+          raise ConnectionError, "#{e.class}: #{e.message}"
+        ensure
+          finish_quietly(response)
+          watchdog.stop
+        end
+        nil
+      end
+
       private
 
       def default_headers
-        {
-          "authorization" => "Bearer #{@api_key}",
-          "content-type" => "application/json",
-          "accept" => "application/json",
-        }
+        # Authorization is optional (I03): sent only when a key is
+        # present and non-empty — a local engine needs none. With a key
+        # the header set is byte-identical to before.
+        if @api_key && !@api_key.empty?
+          {
+            "authorization" => "Bearer #{@api_key}",
+            "content-type" => "application/json",
+            "accept" => "application/json",
+          }
+        else
+          {"content-type" => "application/json", "accept" => "application/json"}
+        end
+      end
+
+      # The deadline knobs (I03): nil = unset (resolution falls through
+      # to the shared timeout); any explicitly-given value must be a
+      # positive Numeric. Applies at the constructor and, via
+      # resolve_deadline, to per-call overrides on raw_stream.
+      def validate_deadline!(name, value)
+        return if value.nil?
+        return if value.is_a?(Numeric) && value.positive?
+
+        raise ArgumentError, "#{name} must be a positive Numeric"
+      end
+
+      # Per-phase deadline resolution for raw_stream (I03): the call
+      # kwarg, else the client's, else the shared timeout. A call kwarg
+      # validates even though the client's own did — both are public
+      # surfaces. Runs BEFORE any reactor dependency, so a bad knob
+      # raises ArgumentError outside a reactor too.
+      def resolve_deadline(name, call_value, client_value)
+        validate_deadline!(name, call_value) unless call_value.nil?
+        call_value || client_value || @timeout
       end
 
       # Session-affinity headers per dialect when a session id is supplied
@@ -156,6 +243,58 @@ module Fast
         end
       end
 
+      # The raw request seam (I05 revision): resolve the payload (a
+      # String verbatim, anything else JSON-generated) and make the ONE
+      # POST, the TTFB deadline armed for creation through status
+      # receipt. The status check stays with the caller — raw_stream's
+      # ensure ladder must see even a non-2xx response to keep the
+      # connection-finish discipline (raise_api_error reads the body;
+      # finish then returns the connection to the pool).
+      def post_raw_body(body, watchdog)
+        payload = body.is_a?(String) ? body : JSON.generate(body)
+        watchdog.arm
+        @client.post(@path, headers: default_headers, body: payload)
+      end
+
+      # The raw each-event seam (I05 revision) — a reusable "each
+      # event" concern: owns the SSE parser, the per-chunk inactivity
+      # re-arm (D5 — the first chunk's arm is the TTFB→inactivity
+      # transition), and the I61 drop-and-count reset; yields the DONE
+      # sentinel in its wire position and every surviving payload
+      # parsed. The block is optional — a blockless call drains.
+      def each_raw_event(response, idle, watchdog, &block)
+        parser = SSE.new
+        @dropped_chunks = 0
+        response.body.each do |chunk|
+          watchdog.arm(idle) # re-arm the inactivity read-timeout per chunk (D5)
+          parser.push(chunk).each do |item|
+            if item.equal?(SSE::DONE) # the wire's terminal token: yielded in place, no payload
+              block&.call(item)
+              next
+            end
+
+            json = parse_chunk(item)
+            next if json.nil? # dropped (I61): counted, the stream continues
+
+            block&.call(json)
+          end
+        end
+      end
+
+      # Cancel→timeout translation (I05 revision): a watchdog fire
+      # surfaces the PHASE-named TimeoutError — headers while the POST
+      # is outstanding (the response is still nil), read inactivity
+      # once the body has begun — while a user's cancel stays
+      # CancelledError (D8). The phase needs no flag: headers are
+      # outstanding exactly until post_raw_body returns.
+      def raise_raw_cancel(watchdog, response, ttfb, idle)
+        if watchdog.fired
+          message = response.nil? ? "no response headers within #{ttfb}s" : "no data within #{idle}s"
+          raise TimeoutError, message
+        end
+        raise CancelledError, "stream cancelled"
+      end
+
       def feed_stream(response, accumulator, watchdog, &block)
         parser = SSE.new
         @dropped_chunks = 0

data/lib/fast/completions/version.rb

@@ -2,6 +2,6 @@
 
 module Fast
   module Completions
-    VERSION = "0.5.0"
+    VERSION = "0.6.0"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fast-completions
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.6.0
 platform: ruby
 authors:
 - Eric Jacobs