fast-completions 0.4.1 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 826572d669198677861fb62faea347d0e74bf99ca3ba39d2e899a159161e762b
-  data.tar.gz: 4e5d12637cc2b273596a38d7cdd39b50ee5f2018a3ba0c18e2c4001a88c32e22
+  metadata.gz: 9d3db818f96038ddf53481fea90ce309a3a01be84d924e8d4707ce121a9232ba
+  data.tar.gz: a6aa0363ad9ba1c6c2b45e47645137390254d8c1b0c8067e8de7f6cc64ed7e43
 SHA512:
-  metadata.gz: 7dde6dc44db9e1993071f6f494a743dad1e89990c67c6b2beeef61638e28cc57aff5af2026b66a42209d4eb27ee8c4848eec559593fbcdcbab1938bf5251d8fc
-  data.tar.gz: '09c28b1d9d980d29abd13a670feba40ec60fb79f3c65246251bc3b1ac042e3ae093728cf3fd2affbadfc09799c97f1f9f4fe59657c535b2376c2eeb0523a024a'
+  metadata.gz: 63a9237cef1bbf316295f96bb6bd13107f5efadef2f0434070070edbd1a74a4668c725d6b7690a73cdfa7d02b750df2c4a85f5a60a1f77b96d0c48d4767c6f20
+  data.tar.gz: 90c846372d3c94697e85de1075d61c7b40525b9c0ad7d2dcc11f632c6204d002e946e5ffb9b62529da0695ebeab930af24413475366ef7d09204f3f233ffefea

data/CHANGELOG.md

@@ -3,6 +3,92 @@
 Notable changes to the fast-completions gem. The format is prose-first,
 one section per released version, newest first.
 
+## 0.6.0 (2026-10-25)
+
+- `Client#raw_stream` — the raw completions surface a proxy needs: the
+  caller's body forwarded verbatim (a String as-is, anything else
+  JSON-generated), every parsed SSE payload yielded to the block in
+  arrival order with the wire's terminal DONE token in its wire
+  position, and `nil` returned — the block is the only output channel.
+  No message accumulation, no tool interpretation, no stop-reason
+  mapping, no salvage, no StreamError, and exactly ONE POST attempt —
+  no auto-retry of any kind; a non-2xx raises the §8 mapping
+  immediately. A malformed payload still rides the I61 chunk seam:
+  one repair pass, then drop-and-count on `dropped_chunks` while the
+  stream continues.
+- The watchdog is now two-deadline: `ttfb_timeout` (status + headers)
+  and `inactivity_timeout` (re-armed after every body chunk), each an
+  optional constructor kwarg and per-call override on `raw_stream`,
+  validated as a positive Numeric, falling back to the shared
+  `timeout` when unset; prefill-scale values (1400 s) are first-class.
+  A fire surfaces the phase-named `TimeoutError` ("no response headers
+  within Ns" / "no data within Ns") — never `CancelledError` — and a
+  user cancel still surfaces `CancelledError`. `Client#stream` keeps
+  its single shared timeout and its byte-identical behavior; the
+  additive default on `Watchdog#arm` is the only existing-signature
+  change.
+- Authentication is optional: `api_key` defaults to nil and the
+  `authorization` header is sent only when the key is present and
+  non-empty — a local engine needs no key. With a key present the
+  header set is unchanged.
+- Docs: a README "Raw streaming" section plus the full reference page
+  (`docs/reference/raw-stream.md`); the README owner-prose gate now
+  rules two zones. Platform unchanged: ruby floor `>= 3.3`, CI matrix
+  3.3/3.4/4.0, no new dependencies.
+- Internal refactor of `Client#raw_stream` (operator review of #7): the
+  method is short orchestration over three private seams — the request
+  seam (payload resolution + the ONE POST under the TTFB arm), the
+  each-event seam (the SSE parser, the per-chunk inactivity re-arm, the
+  DONE yield, the drop-and-count reset), and the cancel→timeout
+  translation — with the timeout phase derived from the response
+  instead of a threading flag. Behavior-identical: signature, yielded
+  events, error classes and messages, no-retry, and the
+  connection-finish discipline are unchanged, and the raw_stream spec
+  passes byte-unmodified.
+
+## 0.5.0 (2026-10-17)
+
+- Tool-args streaming is now an incremental string-state machine
+  (`Fast::Completions::ArgsStreamer`, a new stdlib-only class): instead
+  of eagerly re-parsing the whole arguments scratch on every delta
+  (`JSON.parse` → `repair_json` → re-parse, O(n²) across a stream), each
+  tool-call block owns a streamer whose parser state carries ACROSS
+  deltas and only the NEW fragment's bytes are walked per delta — O(n)
+  total, no per-delta copy of any accumulated buffer, and no exception
+  on the streaming path (the machine is total on any fragment sequence;
+  malformed input freezes the live view rather than crashing the step).
+  This is a deliberate departure from pi, which re-parses per delta
+  (openai-completions.ts :652-658).
+- The live view of a tool-call block's `arguments` Hash is now a
+  first-class feature (operator ruling, plan ENTRY 1 — “watching a
+  file write is extremely nice”): the same Hash object is populated in
+  place for the whole stream, nested containers materialize at their
+  opening token, and an in-flight string value GROWS live — a consumer
+  holding the reference sees it grow — instead of the whole pair
+  vanishing until the closing quote. Escape sequences split by a delta boundary are held (the
+  0.4.0/0.4.1 crash class becomes a structural non-event); invalid
+  escape chars and invalidated `\u` runs contribute their literal text
+  (`repair_json` parity, absorbed rather than called); surrogate halves
+  never enter the live value, pairs combine; scalars materialize as raw
+  text and are replaced by Integer/Float/true/false/null when the token
+  terminates (malformed scalars keep their raw text). After the root
+  object closes — or the stream becomes unparseable — the live view
+  freezes. End states are unchanged and byte-identical to 0.4.1:
+  finalize remains the sole authority, parsing the full scratch once
+  (`parse_streaming_json`: the JSON.parse fast path when the stream
+  completed, the repair walk and `{}` as the last resort otherwise).
+  Mid-stream, the two protocol_spec pins that encoded the old
+  truncation artifact as `eq({})` are amended to the growing view —
+  the ONLY spec changes besides the machine's new unit spec.
+- Bench harness: `compare_refs.rb` now measures a fifth ref — 0.5.0,
+  the working tree at HEAD (the lane tip before the integrator's
+  landing commit) — beside the four releases, pairs 0.4.1→0.5.0, and
+  gains the machine's own pair-scoped floor: tool_args at 60 µs/delta
+  (0.4.1 measured 877.4 µs/delta on the frozen 24 KB / 320-fragment
+  workload; the machine lands in single-digit µs). BASELINE.md is
+  regenerated with the 0.5.0 column. The four bench scripts stay
+  byte-frozen — the workload the floors are calibrated to.
+
 ## 0.4.1 (2026-10-16)
 
 - Fix a mid-stream crash in the tool-call live-parse (regression against

data/README.md

@@ -126,6 +126,23 @@ requests run inside the caller's reactor — the gem never creates its
 own. Cancellation (`Async::Cancel` / `Chat#cancel`) aborts the
 in-flight request, discards the connection, and raises
 `CancelledError` with no phantom terminal event.
+
+## Raw streaming
+
+`Client#raw_stream` is the low-level surface a proxy or gateway needs:
+the request body goes over the wire verbatim (a Hash is JSON-encoded
+as given, a String is sent untouched — no merging, no filtering, no
+`stream` injection), and the block receives every parsed SSE payload
+in arrival order, with the wire's terminal DONE token yielded in its
+wire position. No message accumulation, no tool interpretation, no
+stop-reason mapping; exactly one POST attempt — nothing is retried.
+The watchdog is two-deadline: `ttfb_timeout` bounds the wait for the
+response headers and `inactivity_timeout` bounds the gap between body
+chunks (re-armed after every chunk) — prefill-scale values such as
+1400 s are first-class, and an unset knob falls back to the shared
+`timeout`. The `authorization` header is sent only when `api_key` is
+present and non-empty, so a local engine needs no key. See
+[the raw-stream reference](docs/reference/raw-stream.md).
 [see the tool docs]: docs/reference/tools.md
 [see the dialect docs]: docs/reference/dialects.md
 [Dialect Docs]: docs/reference/dialects.md

data/lib/fast/completions/args_streamer.rb

@@ -0,0 +1,404 @@
+# frozen_string_literal: true
+
+module Fast
+  module Completions
+    # Incremental tool-args string-state machine (I02, 0.5.0). A DELIBERATE
+    # departure from pi: pi re-parses the whole arguments scratch on every
+    # delta (openai-completions.ts :652-658) — O(n²) across a stream and
+    # the pair-in-flight vanishes until its closing quote. This machine
+    # instead carries per-block parser state ACROSS deltas and walks only
+    # the NEW fragment's bytes each time (O(n) total, no per-delta copy of
+    # any accumulated buffer, no exception on the streaming path), growing
+    # the live view (`block[:arguments]`, the root_target below) IN PLACE.
+    # The authoritative whole-buffer parse still happens exactly once, at
+    # finalize (Protocol.parse_streaming_json — the unchanged pi parity).
+    #
+    # THE LIVE-VIEW CONTRACT (frozen, Specification L1-L9). L8 totality:
+    # this source throws no exceptions and installs none — the machine
+    # is total on any fragment sequence: malformed input freezes the
+    # live view and never crashes the stream. The frozen interface:
+    # ArgsStreamer.new(
+    # root_target) — the block's arguments Hash, populated in place (L1);
+    # #consume(String) -> self — append to the scratch, walk the new bytes;
+    # #scratch -> String — the raw bytes accumulated, for the finalize
+    # parse. Everything below the interface is this file's own design.
+    class ArgsStreamer
+      attr_reader :scratch
+
+      # L3 scalar classification (frozen): a scalar that terminates is
+      # REPLACED by Integer/Float/true/false/nil when its raw text is a
+      # WELL-FORMED scalar, and keeps its raw-text String when malformed
+      # — the frozen clause's own examples are `01` and `1.`, so JSON's
+      # grammar (leading zeros, dangling fractions) gates the
+      # classification; the Integer/Float shapes below are the frozen
+      # forms minus the leading-zero overlap the malformed clause rules
+      # out. Integer for -?(0|[1-9]\d*), Float for that mantissa plus a
+      # fraction and/or exponent.
+      INTEGER_TEXT = /\A-?(?:0|[1-9]\d*)\z/
+      FLOAT_TEXT = /\A-?(?:(?:0|[1-9]\d*)\.\d+(?:[eE][+-]?\d+)?|(?:0|[1-9]\d*)(?:\.\d+)?[eE][+-]?\d+)\z/
+
+      # Fast path: a run of string-content bytes needing no per-byte
+      # handling — not a quote, not a backslash, not a raw control char,
+      # not a UTF-8 lead/continuation byte. Binary (/n) so it matches the
+      # byte view of the fragment.
+      PLAIN_RUN = /[^"\\\x00-\x1f\x80-\xff]+/n
+
+      # Byte constants (structure is always ASCII; content may be any
+      # byte — multibyte sequences are assembled, never mistaken for it).
+      QUOTE = 0x22
+      BACKSLASH = 0x5C
+      COLON = 0x3A
+      COMMA = 0x2C
+      OPEN_BRACE = 0x7B
+      CLOSE_BRACE = 0x7D
+      OPEN_BRACKET = 0x5B
+      CLOSE_BRACKET = 0x5D
+
+      def initialize(root_target)
+        @root = root_target
+        @scratch = +""
+        # Machine state (:value/:after_value refer to @stack.last; the
+        # frame layout is [container, :hash|:array]):
+        #   :scan_root  — before the root token (whitespace skipped)
+        #   :frozen     — root closed or structurally unparseable (L7);
+        #                 later bytes never mutate the live view again
+        #   :key_first  — object after `{`: first key or `}`
+        #   :key        — object after a comma: a key is required
+        #   :colon      — between a completed key and its value
+        #   :value      — a value token may start (or `]` closing an
+        #                 EMPTY array, per @may_close)
+        #   :after_value— expecting `,` or the container's closer
+        #   :in_key / :in_string — accumulating a key / a string value
+        #   :in_scalar  — accumulating raw scalar text
+        #   :escape / :u_escape — mid-escape (held across deltas, L5/L6)
+        @state = :scan_root
+        @stack = []
+        @key = nil        # completed key text for the pending pair
+        @buf = nil        # the String being grown (a live value or a key)
+        @str_back = nil   # :in_key or :in_string — where an escape returns
+        @cell = nil       # [container, key_or_index] — the in-flight scalar slot
+        @scalar = nil     # raw scalar text (replaced on termination, L3)
+        @digits = nil     # collected hex digits of a \u run
+        @pending_high = nil # held high surrogate (L6)
+        @utf8_bytes = nil   # partial multibyte sequence being assembled
+        @utf8_len = 0
+        @may_close = false # :value may see `]` (freshly opened array)
+      end
+
+      # Append the fragment to the scratch and walk ONLY its bytes. An
+      # empty fragment is a no-op (L8); a frozen machine still accumulates
+      # the scratch — finalize, not this walk, is the authority (L9).
+      def consume(fragment)
+        return self unless fragment.is_a?(String)
+
+        # Scratch discipline (L8 totality — no exception ever leaves this
+        # file): the scratch stays UTF-8 so the finalize parse gets a
+        # JSON-parseable string. A same-encoding or ASCII-only fragment
+        # appends with no copy; anything else (a binary fragment cut
+        # mid-multibyte, odd encodings) is normalized byte-for-byte to
+        # UTF-8 first — invalid bytes stay invalid and finalize's repair
+        # path owns them. `bytes` is the walk view: binary for the
+        # byte-level regex, the fragment itself when it is ASCII-only
+        # (the regex accepts an ASCII-only string in any encoding).
+        bytes = fragment.ascii_only? ? fragment : fragment.b
+        if fragment.encoding == Encoding::UTF_8 || fragment.ascii_only?
+          @scratch << fragment
+        else
+          @scratch << bytes.dup.force_encoding(Encoding::UTF_8)
+        end
+        return self if @state == :frozen
+
+        frag = bytes
+        i = 0
+        length = frag.bytesize
+        while i < length
+          byte = frag.getbyte(i)
+          if ((@state == :in_string || @state == :in_key) && @utf8_bytes.nil?) &&
+             byte != QUOTE && byte != BACKSLASH && byte >= 0x20 && byte < 0x80
+            # Fast path: a plain run — one in-place append, one regex scan
+            # of the NEW bytes only. A held high surrogate drops here: its
+            # pair never lands (L6).
+            @pending_high = nil
+            run = PLAIN_RUN.match(frag, i)[0]
+            @buf << run
+            i += run.bytesize
+            next
+          end
+          i += 1
+          case @state
+          when :in_string, :in_key
+            consume_string_byte(byte) || (i -= 1) # false = reprocess the byte
+          when :escape
+            consume_escape_byte(byte) || (i -= 1)
+          when :u_escape
+            consume_unicode_byte(byte) || (i -= 1)
+          when :scan_root
+            case byte
+            when 0x09, 0x0A, 0x0D, 0x20 then nil # whitespace skipped
+            when OPEN_BRACE
+              @stack.push([@root, :hash]) # the live Hash populates (L7)
+              @state = :key_first
+            else # array, scalar, garbage before `{` — empty and frozen
+              @state = :frozen
+            end
+          when :key_first, :key
+            case byte
+            when 0x09, 0x0A, 0x0D, 0x20 then nil
+            when QUOTE
+              @buf = +""
+              @pending_high = nil
+              @utf8_bytes = nil
+              @state = :in_key
+            when CLOSE_BRACE
+              if @state == :key_first
+                close_container # `{}` — the empty object closes
+              else # after a comma a key is required: unparseable
+                @state = :frozen
+              end
+            else
+              @state = :frozen
+            end
+          when :colon
+            case byte
+            when 0x09, 0x0A, 0x0D, 0x20 then nil
+            when COLON
+              @may_close = false
+              @state = :value
+            else
+              @state = :frozen
+            end
+          when :value
+            case byte
+            when 0x09, 0x0A, 0x0D, 0x20 then nil
+            when QUOTE then open_string_value
+            when OPEN_BRACE then open_container({}, :hash)
+            when OPEN_BRACKET then open_container([], :array)
+            when CLOSE_BRACKET
+              if @may_close # `[]`; any other `]` here is unparseable
+                close_container
+              else
+                @state = :frozen
+              end
+            when CLOSE_BRACE, COMMA then @state = :frozen # no value can start with these
+            else
+              open_scalar(byte)
+            end
+          when :after_value
+            case byte
+            when 0x09, 0x0A, 0x0D, 0x20 then nil
+            when COMMA
+              if @stack.last[1] == :hash
+                @state = :key
+              else
+                @may_close = false
+                @state = :value
+              end
+            when CLOSE_BRACE
+              @stack.last[1] == :hash ? close_container : @state = :frozen
+            when CLOSE_BRACKET
+              @stack.last[1] == :array ? close_container : @state = :frozen
+            else
+              @state = :frozen
+            end
+          when :in_scalar
+            case byte
+            when COMMA, CLOSE_BRACE, CLOSE_BRACKET, 0x09, 0x0A, 0x0D, 0x20
+              # Token terminated (delimiter, closer, or whitespace): the
+              # raw-text String is REPLACED in its slot by the classified
+              # value (L3); the terminator itself is reprocessed below.
+              @cell[0][@cell[1]] = scalar_value(@scalar)
+              @state = :after_value
+              i -= 1
+            else
+              @scalar << byte
+            end
+          end
+        end
+        self
+      end
+
+      private
+
+      # One byte inside a key or a string value. Returns false when the
+      # byte must be reprocessed (a broken multibyte sequence is dropped
+      # and the offending byte starts over).
+      def consume_string_byte(byte)
+        if @utf8_bytes
+          if byte >= 0x80 && byte <= 0xBF
+            @utf8_bytes << byte
+            if @utf8_bytes.size == @utf8_len
+              sequence = @utf8_bytes.pack("C*").force_encoding(Encoding::UTF_8)
+              @buf << sequence if sequence.valid_encoding?
+              @utf8_bytes = nil
+            end
+          else # the sequence never completes: dropped, byte reprocessed
+            @utf8_bytes = nil
+            return false
+          end
+        elsif byte >= 0x80
+          @pending_high = nil # a held high's pair never lands (L6)
+          if byte >= 0xC2 && byte <= 0xF4 # a valid lead byte; anything else is dropped
+            @utf8_len = byte < 0xE0 ? 2 : (byte < 0xF0 ? 3 : 4)
+            @utf8_bytes = [byte]
+          end
+        elsif byte == QUOTE # the string closes
+          @pending_high = nil # a held high's pair never lands (L6)
+          if @state == :in_key
+            @key = @buf
+            @state = :colon
+          else
+            @state = :after_value
+          end
+        elsif byte == BACKSLASH
+          @str_back = @state
+          @state = :escape
+        else # plain ASCII content, raw control characters included (L5)
+          @pending_high = nil
+          @buf << byte
+        end
+        true
+      end
+
+      # The byte after a backslash. Returns false when the byte must be
+      # reprocessed (an invalid escape char contributes the literal
+      # backslash, then the char itself is handled as content, L5).
+      def consume_escape_byte(byte)
+        case byte
+        when 0x75 # u — the hex run starts; a held high may still pair (L6)
+          @digits = +""
+          @state = :u_escape
+        when QUOTE then settle_escape("\"")
+        when BACKSLASH then settle_escape("\\")
+        when 0x2F then settle_escape("/") # \/ — repair_json parity
+        when 0x62 then settle_escape("\b")
+        when 0x66 then settle_escape("\f")
+        when 0x6E then settle_escape("\n")
+        when 0x72 then settle_escape("\r")
+        when 0x74 then settle_escape("\t")
+        else
+          @pending_high = nil
+          @buf << "\\" # invalid escape char: literal `\`, char reprocessed
+          @state = @str_back
+          return false
+        end
+        true
+      end
+
+      def settle_escape(text)
+        @pending_high = nil # a held high's pair never lands (L6)
+        @buf << text
+        @state = @str_back
+      end
+
+      # One byte of a \u run. Returns false when the byte must be
+      # reprocessed (an invalidated run contributes literal `\u` + the
+      # digits collected, then the offending byte is handled as content,
+      # L5 — a short run invalidated by the string's closing quote is the
+      # "short at close" case).
+      def consume_unicode_byte(byte)
+        if hex_byte?(byte)
+          @digits << byte
+          if @digits.size == 4
+            codepoint = @digits.to_i(16)
+            if codepoint >= 0xD800 && codepoint <= 0xDBFF
+              @pending_high = codepoint # held for a possible pair (L6)
+            elsif codepoint >= 0xDC00 && codepoint <= 0xDFFF
+              if @pending_high # the pair combines into one codepoint (L6)
+                pair = 0x10000 + ((@pending_high & 0x3FF) << 10) + (codepoint & 0x3FF)
+                @buf << pair
+                @pending_high = nil
+              end # a lone low half never enters the live value (L6)
+            else
+              @pending_high = nil
+              @buf << codepoint
+            end
+            @state = @str_back
+          end
+        else
+          @pending_high = nil
+          @buf << "\\u"
+          @buf << @digits
+          @state = @str_back
+          return false
+        end
+        true
+      end
+
+      def hex_byte?(byte)
+        (byte >= 0x30 && byte <= 0x39) || (byte >= 0x41 && byte <= 0x46) ||
+          (byte >= 0x61 && byte <= 0x66)
+      end
+
+      # A string value materializes at its OPENING quote — the slot exists
+      # with "" before any content byte (L3) — and grows by in-place
+      # append for the rest of the stream (L1).
+      def open_string_value
+        value = +""
+        attach(value)
+        @buf = value
+        @pending_high = nil
+        @utf8_bytes = nil
+        @state = :in_string
+      end
+
+      # An object/array value materializes at its opening token and is
+      # inserted into its parent immediately (L3), then mutated in place
+      # (L1). A fresh array may still be closed empty while @may_close.
+      def open_container(container, kind)
+        attach(container)
+        @stack.push([container, kind])
+        if kind == :hash
+          @state = :key_first
+        else
+          @may_close = true
+          @state = :value
+        end
+      end
+
+      # A scalar materializes as its raw-text String at its FIRST byte
+      # (L3), inserted into its parent; the slot is replaced when the
+      # token terminates (see :in_scalar above).
+      def open_scalar(byte)
+        @scalar = +""
+        @scalar << byte
+        attach(@scalar)
+        @state = :in_scalar
+      end
+
+      # Insert a fresh value into the innermost container. Duplicate keys
+      # collapse to one slot, last value wins (L2); array elements append.
+      # The slot pair also serves scalar replacement — `container[key]=`
+      # and `container[index]=` are the same expression for both kinds.
+      def attach(value)
+        frame = @stack.last
+        if frame[1] == :hash
+          frame[0][@key] = value
+          @cell = [frame[0], @key]
+        else
+          @cell = [frame[0], frame[0].size]
+          frame[0] << value
+        end
+      end
+
+      # Close the innermost container. The root object closing (or the
+      # last frame popping empty) freezes the live view (L7).
+      def close_container
+        @stack.pop
+        @state = @stack.empty? ? :frozen : :after_value
+      end
+
+      def scalar_value(text)
+        case text
+        when "true" then true
+        when "false" then false
+        when "null" then nil
+        else
+          if text.match?(INTEGER_TEXT) then text.to_i
+          elsif text.match?(FLOAT_TEXT) then text.to_f
+          else text # malformed (`01`, `1.` at close): keeps its raw text
+          end
+        end
+      end
+    end
+  end
+end

data/lib/fast/completions/client.rb

@@ -34,11 +34,14 @@ module Fast
         # resets with the timer: a retried attempt (I62) must not inherit
         # the previous attempt's fire, or the post-stream guard and the
         # cancel-vs-fired discipline would misread attempt 2 (D5/D8).
-        def arm
+        # The additive seconds parameter (I03) lets one watchdog arm
+        # deadlines of different lengths — raw_stream's split phases —
+        # while a bare arm keeps the constructor value (stream, unchanged).
+        def arm(seconds = @seconds)
           stop
           @fired = false
           @timer = Async do
-            sleep @seconds
+            sleep seconds
             @fired = true
             @task.cancel
           end
@@ -55,15 +58,20 @@ module Fast
         Errno::ENETUNREACH, Errno::ETIMEDOUT, EOFError, IOError, SocketError,
       ].freeze
 
-      attr_reader :timeout, :max_retries, :dropped_chunks
+      attr_reader :timeout, :max_retries, :dropped_chunks, :ttfb_timeout, :inactivity_timeout
 
-      def initialize(base_url:, api_key:, timeout: DEFAULT_TIMEOUT, max_retries: DEFAULT_MAX_RETRIES)
+      def initialize(base_url:, api_key: nil, timeout: DEFAULT_TIMEOUT, max_retries: DEFAULT_MAX_RETRIES,
+        ttfb_timeout: nil, inactivity_timeout: nil)
         raise ArgumentError, "max_retries must be an Integer >= 0" unless max_retries.is_a?(Integer) && max_retries >= 0
+        validate_deadline!(:ttfb_timeout, ttfb_timeout)
+        validate_deadline!(:inactivity_timeout, inactivity_timeout)
 
         @endpoint = Async::HTTP::Endpoint.parse(base_url)
         @api_key = api_key
         @timeout = timeout
         @max_retries = max_retries
+        @ttfb_timeout = ttfb_timeout
+        @inactivity_timeout = inactivity_timeout
         @client = Async::HTTP::Client.new(@endpoint)
         @path = @endpoint.path.chomp("/") + "/chat/completions"
       end
@@ -131,14 +139,93 @@ module Fast
         deliver_terminal(accumulator, &block)
       end
 
+      # The raw surface a proxy needs (I03, BRIEF §3): the caller's
+      # request body forwarded VERBATIM (a String as-is, anything else
+      # JSON.generate'd — no merging, no filtering, no stream
+      # injection, no Model/dialect involvement, no session-affinity
+      # headers), every parsed SSE data payload yielded to the block in
+      # arrival order — the SSE::DONE sentinel in its wire position —
+      # and nil returned: the block is the only output channel. No
+      # Message accumulation, no tool interpretation, no stop-reason
+      # mapping, no salvage, no StreamError — a stream that ends without
+      # the terminal token simply ends. Exactly ONE POST attempt: no
+      # create_response, no retry logic, no x-should-retry consultation;
+      # a non-2xx raises the existing §8 mapping (raise_api_error).
+      #
+      # Watchdog, split by I03: two deadlines. TTFB (status + headers)
+      # arms before the POST; from response receipt on the inactivity
+      # deadline re-arms after every body chunk (D5). Each resolves per
+      # phase — call kwarg > client kwarg > the shared timeout. A fire
+      # cancels the waiting task and surfaces the phase-named
+      # TimeoutError, never CancelledError (D8); a user cancel surfaces
+      # CancelledError. Runs inside the caller's task (§6).
+      #
+      # I05 revision (operator review of #7 — "the raw_stream method
+      # is way too long and not idiomatic ruby"): the method is now
+      # short orchestration over three private seams — the request seam
+      # (post_raw_body), the each-event seam (each_raw_event), and the
+      # cancel translation (raise_raw_cancel). The phase a timeout
+      # fires in is derived from the response itself — headers are
+      # outstanding exactly until the POST returns — so no phase flag
+      # threads through the ladder.
+      def raw_stream(body, ttfb_timeout: nil, inactivity_timeout: nil, &block)
+        ttfb = resolve_deadline(:ttfb_timeout, ttfb_timeout, @ttfb_timeout)
+        idle = resolve_deadline(:inactivity_timeout, inactivity_timeout, @inactivity_timeout)
+
+        watchdog = Watchdog.new(Async::Task.current, ttfb)
+        response = nil
+        begin
+          response = post_raw_body(body, watchdog)
+          raise_api_error(response) unless (200...300).cover?(response.status)
+
+          each_raw_event(response, idle, watchdog, &block)
+        rescue Async::Cancel
+          raise_raw_cancel(watchdog, response, ttfb, idle)
+        rescue *NETWORK_ERRORS => e
+          raise ConnectionError, "#{e.class}: #{e.message}"
+        ensure
+          finish_quietly(response)
+          watchdog.stop
+        end
+        nil
+      end
+
       private
 
       def default_headers
-        {
-          "authorization" => "Bearer #{@api_key}",
-          "content-type" => "application/json",
-          "accept" => "application/json",
-        }
+        # Authorization is optional (I03): sent only when a key is
+        # present and non-empty — a local engine needs none. With a key
+        # the header set is byte-identical to before.
+        if @api_key && !@api_key.empty?
+          {
+            "authorization" => "Bearer #{@api_key}",
+            "content-type" => "application/json",
+            "accept" => "application/json",
+          }
+        else
+          {"content-type" => "application/json", "accept" => "application/json"}
+        end
+      end
+
+      # The deadline knobs (I03): nil = unset (resolution falls through
+      # to the shared timeout); any explicitly-given value must be a
+      # positive Numeric. Applies at the constructor and, via
+      # resolve_deadline, to per-call overrides on raw_stream.
+      def validate_deadline!(name, value)
+        return if value.nil?
+        return if value.is_a?(Numeric) && value.positive?
+
+        raise ArgumentError, "#{name} must be a positive Numeric"
+      end
+
+      # Per-phase deadline resolution for raw_stream (I03): the call
+      # kwarg, else the client's, else the shared timeout. A call kwarg
+      # validates even though the client's own did — both are public
+      # surfaces. Runs BEFORE any reactor dependency, so a bad knob
+      # raises ArgumentError outside a reactor too.
+      def resolve_deadline(name, call_value, client_value)
+        validate_deadline!(name, call_value) unless call_value.nil?
+        call_value || client_value || @timeout
       end
 
       # Session-affinity headers per dialect when a session id is supplied
@@ -156,6 +243,58 @@ module Fast
         end
       end
 
+      # The raw request seam (I05 revision): resolve the payload (a
+      # String verbatim, anything else JSON-generated) and make the ONE
+      # POST, the TTFB deadline armed for creation through status
+      # receipt. The status check stays with the caller — raw_stream's
+      # ensure ladder must see even a non-2xx response to keep the
+      # connection-finish discipline (raise_api_error reads the body;
+      # finish then returns the connection to the pool).
+      def post_raw_body(body, watchdog)
+        payload = body.is_a?(String) ? body : JSON.generate(body)
+        watchdog.arm
+        @client.post(@path, headers: default_headers, body: payload)
+      end
+
+      # The raw each-event seam (I05 revision) — a reusable "each
+      # event" concern: owns the SSE parser, the per-chunk inactivity
+      # re-arm (D5 — the first chunk's arm is the TTFB→inactivity
+      # transition), and the I61 drop-and-count reset; yields the DONE
+      # sentinel in its wire position and every surviving payload
+      # parsed. The block is optional — a blockless call drains.
+      def each_raw_event(response, idle, watchdog, &block)
+        parser = SSE.new
+        @dropped_chunks = 0
+        response.body.each do |chunk|
+          watchdog.arm(idle) # re-arm the inactivity read-timeout per chunk (D5)
+          parser.push(chunk).each do |item|
+            if item.equal?(SSE::DONE) # the wire's terminal token: yielded in place, no payload
+              block&.call(item)
+              next
+            end
+
+            json = parse_chunk(item)
+            next if json.nil? # dropped (I61): counted, the stream continues
+
+            block&.call(json)
+          end
+        end
+      end
+
+      # Cancel→timeout translation (I05 revision): a watchdog fire
+      # surfaces the PHASE-named TimeoutError — headers while the POST
+      # is outstanding (the response is still nil), read inactivity
+      # once the body has begun — while a user's cancel stays
+      # CancelledError (D8). The phase needs no flag: headers are
+      # outstanding exactly until post_raw_body returns.
+      def raise_raw_cancel(watchdog, response, ttfb, idle)
+        if watchdog.fired
+          message = response.nil? ? "no response headers within #{ttfb}s" : "no data within #{idle}s"
+          raise TimeoutError, message
+        end
+        raise CancelledError, "stream cancelled"
+      end
+
       def feed_stream(response, accumulator, watchdog, &block)
         parser = SSE.new
         @dropped_chunks = 0

data/lib/fast/completions/protocol.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require_relative "args_streamer"
+
 module Fast
   module Completions
     # THE BYTE CONTRACT (Specification §9): Ruby Hash insertion order +
@@ -311,7 +313,7 @@ module Fast
           @thinking_block = nil
           @tool_by_index = {}
           @tool_by_id = {}
-          @args_scratch = {}.compare_by_identity
+          @args_streamers = {}.compare_by_identity # per-block ArgsStreamer (I02); each owns its scratch
           @streamed_details = nil # :334-341
           @has_finish_reason = false
           @stop_reason = nil
@@ -440,7 +442,7 @@ module Fast
           # its block — by index, by id, or by creation — while CARRYING an
           # index the block has never bound, binds it here. FIRST index
           # wins; one index per block, ever. Boundness is tracked
-          # accumulator-side (identity-keyed, like @args_scratch): the
+          # accumulator-side (identity-keyed, like @args_streamers): the
           # blocks ARE message content — no scratch keys on them.
           @tool_index_bound ||= {}.compare_by_identity
           if index && !@tool_index_bound.key?(block)
@@ -450,8 +452,15 @@ module Fast
 
           raw = entry.dig("function", "arguments")
           raw = "" unless raw.is_a?(String)
-          @args_scratch[block] = (@args_scratch[block] || "") + raw
-          block[:arguments] = Protocol.parse_streaming_json(@args_scratch[block]) # best-effort live parse (OQ-4)
+          # I02 (0.5.0): the incremental args machine. The old seam — a
+          # full-buffer copy plus an eager whole-scratch re-parse per delta
+          # — is gone. This is a DELIBERATE departure from pi, which
+          # re-parses per delta (openai-completions.ts :652-658): the
+          # streamer carries parser state across deltas, walks ONLY the
+          # new fragment's bytes, and grows the live view in place (the
+          # block's arguments Hash); the one authoritative whole-buffer
+          # parse still happens at finalize (finalize_blocks below).
+          (@args_streamers[block] ||= ArgsStreamer.new(block[:arguments])).consume(raw)
           events << Event.new(type: :tool_call_delta, content_index: content_index(block),
             delta: raw, partial: @partial)
           events
@@ -469,7 +478,13 @@ module Fast
               block[:signature] = JSON.generate(@streamed_details) if @streamed_details
               events << Event.new(type: :thinking_end, content_index: index, content: block[:thinking], partial: @partial)
             when :tool_call
-              raw = @args_scratch.delete(block) || ""
+              streamer = @args_streamers.delete(block)
+              raw = streamer ? streamer.scratch : "" # no-fragment blocks finalize an empty scratch
+              # Finalize is the SOLE authority (L9): one whole-buffer parse
+              # of the raw bytes — the JSON.parse fast path when the stream
+              # completed, the repair walk + {} last resort otherwise
+              # (pi :453 / utils/json-parse.ts :20-90). The live Hash the
+              # machine grew is replaced here, exactly once.
               block[:arguments] = Protocol.parse_streaming_json(raw)
               events << Event.new(type: :tool_call_end, content_index: index,
                 tool_call: {id: block[:id], name: block[:name], arguments: block[:arguments]},

data/lib/fast/completions/version.rb

@@ -2,6 +2,6 @@
 
 module Fast
   module Completions
-    VERSION = "0.4.1"
+    VERSION = "0.6.0"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fast-completions
 version: !ruby/object:Gem::Version
-  version: 0.4.1
+  version: 0.6.0
 platform: ruby
 authors:
 - Eric Jacobs
@@ -34,6 +34,7 @@ files:
 - CHANGELOG.md
 - README.md
 - lib/fast/completions.rb
+- lib/fast/completions/args_streamer.rb
 - lib/fast/completions/cancellation.rb
 - lib/fast/completions/chat.rb
 - lib/fast/completions/client.rb