telemetry-dev-anthropic 0.2.2__tar.gz → 0.2.4__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: telemetry-dev-anthropic
3
- Version: 0.2.2
3
+ Version: 0.2.4
4
4
  Summary: Anthropic integration for telemetry.dev Python SDK
5
5
  Keywords: telemetry,opentelemetry,anthropic,claude,llm,genai,tracing
6
6
  Author: telemetry.dev
@@ -96,15 +96,14 @@ The integration maps native Anthropic request and response shapes directly into
96
96
 
97
97
  Native Anthropic stream events pass through unmodified. The span records time to first chunk on the first received event, merges usage from `message_start` and `message_delta`, aggregates text, tool-use JSON, and thinking blocks, and ends on stream exhaustion, close, context-manager exit, or error.
98
98
 
99
- Reconstructed output is bounded to 48 KiB and 1,000 items. The configured `max_attribute_length` does not shrink these retention bounds; the core applies it to the exported attribute after the mask runs, with its `...[truncated]` marker, without setting `telemetry.dev.capture.truncated`. A stream that ends without `message_stop`, encounters a mapping failure,
99
+ Reconstructed output is kept up to 16 MiB, and all streams in flight share the `max_stream_bytes` bound of the core (64 MiB by default). These bounds need a `telemetry-dev` core with `max_stream_bytes`; with an older core, each stream keeps up to 64 KiB and 1,024 items. The configured `max_attribute_length` does not shrink these retention bounds; the core applies it to the exported attribute after the mask runs, with its `...[truncated]` marker, without setting `telemetry.dev.capture.truncated`. A stream that ends without `message_stop`, encounters a mapping failure,
100
100
  or exceeds a capture bound sets `telemetry.dev.capture.truncated`. With a mask configured,
101
101
  incomplete output is omitted because the mask cannot inspect the complete value. When output
102
102
  capture is disabled, response IDs and models, finish reasons, usage, timing, and errors may still
103
103
  be recorded.
104
104
  Capture flags do not gate stop sequences or caller-supplied metadata or raw attributes; redact those
105
105
  separately when needed. Exception messages and stack traces are recorded only when both
106
- `capture_input` and `capture_output` are on, and pass through `mask`. Older `telemetry-dev`
107
- versions record them regardless of capture flags. Request tools and `tool_choice` are part of the captured input, so `capture_input=False` removes them. Each tool's type and name are still recorded in `gen_ai.tool.definitions`, and `mask` does not apply to them; descriptions and input schemas are not.
106
+ `capture_input` and `capture_output` are on, and pass through `mask`. Request tools and `tool_choice` are part of the captured input, so `capture_input=False` removes them. Each tool's type and name are still recorded in `gen_ai.tool.definitions`, and `mask` does not apply to them; descriptions and input schemas are not.
108
107
 
109
108
  `messages.stream()` starts the span when the context manager is entered, because that is when the Anthropic SDK opens the HTTP stream.
110
109
 
@@ -73,15 +73,14 @@ The integration maps native Anthropic request and response shapes directly into
73
73
 
74
74
  Native Anthropic stream events pass through unmodified. The span records time to first chunk on the first received event, merges usage from `message_start` and `message_delta`, aggregates text, tool-use JSON, and thinking blocks, and ends on stream exhaustion, close, context-manager exit, or error.
75
75
 
76
- Reconstructed output is bounded to 48 KiB and 1,000 items. The configured `max_attribute_length` does not shrink these retention bounds; the core applies it to the exported attribute after the mask runs, with its `...[truncated]` marker, without setting `telemetry.dev.capture.truncated`. A stream that ends without `message_stop`, encounters a mapping failure,
76
+ Reconstructed output is kept up to 16 MiB, and all streams in flight share the `max_stream_bytes` bound of the core (64 MiB by default). These bounds need a `telemetry-dev` core with `max_stream_bytes`; with an older core, each stream keeps up to 64 KiB and 1,024 items. The configured `max_attribute_length` does not shrink these retention bounds; the core applies it to the exported attribute after the mask runs, with its `...[truncated]` marker, without setting `telemetry.dev.capture.truncated`. A stream that ends without `message_stop`, encounters a mapping failure,
77
77
  or exceeds a capture bound sets `telemetry.dev.capture.truncated`. With a mask configured,
78
78
  incomplete output is omitted because the mask cannot inspect the complete value. When output
79
79
  capture is disabled, response IDs and models, finish reasons, usage, timing, and errors may still
80
80
  be recorded.
81
81
  Capture flags do not gate stop sequences or caller-supplied metadata or raw attributes; redact those
82
82
  separately when needed. Exception messages and stack traces are recorded only when both
83
- `capture_input` and `capture_output` are on, and pass through `mask`. Older `telemetry-dev`
84
- versions record them regardless of capture flags. Request tools and `tool_choice` are part of the captured input, so `capture_input=False` removes them. Each tool's type and name are still recorded in `gen_ai.tool.definitions`, and `mask` does not apply to them; descriptions and input schemas are not.
83
+ `capture_input` and `capture_output` are on, and pass through `mask`. Request tools and `tool_choice` are part of the captured input, so `capture_input=False` removes them. Each tool's type and name are still recorded in `gen_ai.tool.definitions`, and `mask` does not apply to them; descriptions and input schemas are not.
85
84
 
86
85
  `messages.stream()` starts the span when the context manager is entered, because that is when the Anthropic SDK opens the HTTP stream.
87
86
 
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "telemetry-dev-anthropic"
3
- version = "0.2.2"
3
+ version = "0.2.4"
4
4
  description = "Anthropic integration for telemetry.dev Python SDK"
5
5
  readme = "README.md"
6
6
  license = "MIT"
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "telemetry-dev-anthropic"
3
- version = "0.2.2"
3
+ version = "0.2.4"
4
4
  description = "Anthropic integration for telemetry.dev Python SDK"
5
5
  readme = "README.md"
6
6
  license = "MIT"
@@ -2,12 +2,14 @@ from __future__ import annotations
2
2
 
3
3
  import importlib
4
4
  import json
5
+ import os
5
6
  import re
6
7
  import threading
7
8
  import time
8
9
  import types
9
10
  from collections.abc import AsyncIterator, Callable, Iterable, Iterator, Mapping, Sequence
10
11
  from functools import wraps
12
+ from io import StringIO
11
13
  from typing import Any, Final, TypeVar, cast
12
14
 
13
15
  import anthropic
@@ -18,7 +20,7 @@ from anthropic.resources.beta.messages import Messages as BetaMessages
18
20
  from anthropic.resources.messages import AsyncMessages, Messages
19
21
  from telemetry_dev._serialize import json_dumps
20
22
 
21
- __version__ = "0.2.2"
23
+ __version__ = "0.2.4"
22
24
 
23
25
  ProviderResolver = Callable[[object | None], str]
24
26
 
@@ -27,6 +29,22 @@ _ORIGINAL_ATTR = "_telemetry_dev_anthropic_original"
27
29
  _ORIGINALS: list[tuple[type[Any], str, Any, Any]] = []
28
30
  _installed = False
29
31
  _install_lock = threading.Lock()
32
+
33
+
34
+ def _reset_install_lock_in_child() -> None:
35
+ """A thread of the parent can hold the lock at a fork, and only this thread exists here."""
36
+ global _install_lock
37
+ _install_lock = threading.Lock()
38
+
39
+
40
+ # A hook is enough: a fork that runs no hooks, as uWSGI's fork of its workers by default,
41
+ # starts from its master, which serves no requests. os.register_at_fork cannot remove a hook,
42
+ # and importlib.reload runs this module again with the same globals, so the hook of the first
43
+ # run stays. Windows has no fork.
44
+ if hasattr(os, "register_at_fork") and "_install_fork_hook" not in globals():
45
+ _install_fork_hook = True
46
+ os.register_at_fork(after_in_child=_reset_install_lock_in_child)
47
+
30
48
  _T = TypeVar("_T")
31
49
  _OMIT = object()
32
50
  # Each of these issues its own request: parse() and stream() do not route through create().
@@ -484,29 +502,144 @@ def _end_once(handle: telemetry_dev.SpanHandle) -> Callable[..., None]:
484
502
  return end
485
503
 
486
504
 
505
+ def _stream_budget() -> telemetry_dev.CaptureBudget:
506
+ # Older cores have no for_stream and bound each budget at 64 KiB on its own.
507
+ for_stream = getattr(telemetry_dev.CaptureBudget, "for_stream", None)
508
+ budget = for_stream() if callable(for_stream) else telemetry_dev.CaptureBudget()
509
+ return cast(telemetry_dev.CaptureBudget, budget)
510
+
511
+
512
+ def _reserve_bytes(budget: telemetry_dev.CaptureBudget, total: int) -> bool:
513
+ # Older cores have no reserve_bytes; there the caller's check against max_bytes stands.
514
+ reserve = getattr(budget, "reserve_bytes", None)
515
+ if callable(reserve):
516
+ return bool(reserve(total))
517
+ budget.bytes_used = total
518
+ return True
519
+
520
+
521
+ def _release_budget(budget: telemetry_dev.CaptureBudget) -> None:
522
+ release = getattr(budget, "release", None)
523
+ if callable(release):
524
+ release()
525
+
526
+
527
+ # On Python 3.10 and 3.11, StringIO keeps up to 100,000 written strings in a list before it
528
+ # joins them, about 60 bytes each that the stream bound does not charge: 15 bytes a character
529
+ # for 4-character deltas. Deltas are joined 256 at a time, so that those waiting hold under
530
+ # about 22 KB, onto a string that is written once it has 4,096 characters, so that what is
531
+ # written costs about 1.5% more than its text. Python 3.12 dropped the list.
532
+ _DELTA_BATCH = 256
533
+ _WRITE_CHARS = 4096
534
+
535
+
536
+ class _DeltaBuffer:
537
+ # A write takes several steps, unlike one StringIO call, so without the lock a read on
538
+ # another thread, such as one that closes the stream, could copy a batch twice or drop it.
539
+ # A fork copies the lock held if another thread was writing, and that thread does not run
540
+ # in the child to release it. Every fork, also one that runs no at-fork hooks, changes the
541
+ # process ID, so a child takes a new lock. The check is inline because it runs per delta.
542
+ # A signal handler runs on the thread it interrupts, so one that reads the stream, such as
543
+ # one that closes it, can read inside a write. So the lock is reentrant, and each step takes
544
+ # the deltas and the tail out of the buffer before it joins or writes them, so that a read
545
+ # inside the step does not write them as well; that read returns a prefix, which can leave
546
+ # out the deltas being joined. A handler that writes inside a step is not supported. An
547
+ # exception inside a step, such as KeyboardInterrupt from a handler, puts back the text the
548
+ # step took unless the StringIO has it, so that a later read still has every delta.
549
+ __slots__ = ("_buffer", "_lock", "_pending", "_pid", "_tail")
550
+
551
+ def __init__(self) -> None:
552
+ self._buffer = StringIO()
553
+ self._lock = threading.RLock()
554
+ self._pid = os.getpid()
555
+ self._pending: list[str] = []
556
+ self._tail = ""
557
+
558
+ def write(self, text: str) -> None:
559
+ pid = os.getpid()
560
+ if self._pid != pid:
561
+ self._lock = threading.RLock()
562
+ self._pid = pid
563
+ with self._lock:
564
+ self._pending.append(text)
565
+ if len(self._pending) >= _DELTA_BATCH:
566
+ self._join(_WRITE_CHARS)
567
+
568
+ def getvalue(self) -> str:
569
+ pid = os.getpid()
570
+ if self._pid != pid:
571
+ self._lock = threading.RLock()
572
+ self._pid = pid
573
+ with self._lock:
574
+ self._join(1)
575
+ return self._buffer.getvalue()
576
+
577
+ def _join(self, write_at: int) -> None:
578
+ if not self._pending and len(self._tail) < write_at:
579
+ return
580
+ written = self._buffer.tell()
581
+ pending: list[str] = []
582
+ tail = ""
583
+ try:
584
+ pending, self._pending = self._pending, []
585
+ tail, self._tail = self._tail, ""
586
+ joined = "".join([tail, *pending])
587
+ if len(joined) >= write_at:
588
+ self._buffer.write(joined)
589
+ else:
590
+ self._tail = joined
591
+ except BaseException:
592
+ if self._buffer.tell() == written:
593
+ self._pending[:0] = pending
594
+ self._tail = tail + self._tail
595
+ raise
596
+
597
+
487
598
  class _StreamState:
488
599
  def __init__(self, capture_output: bool = True) -> None:
489
600
  self.capture_output = capture_output
490
601
  self.message_stopped = False
491
- self.blocks: dict[int, dict[str, Any]] = {}
492
- self.tool_json: dict[int, str] = {}
602
+ self._blocks: dict[int, dict[str, Any]] = {}
603
+ self._tool_json: dict[int, str] = {}
604
+ # Appended deltas wait here and are joined when the blocks are next read: adding each
605
+ # delta to the string copies all the text before it, which is quadratic in a long
606
+ # stream. Only the appending path reads _blocks and _tool_json directly.
607
+ self._appended_text: dict[tuple[int, str], _DeltaBuffer] = {}
608
+ self._appended_json: dict[int, _DeltaBuffer] = {}
493
609
  self.usage: dict[str, int | float] | None = None
494
610
  self.one_hour_cache_creation_tokens: int | float | None = None
495
611
  self.iterations: _IterationsUsage | None = None
496
612
  self.finish_reason: str | None = None
497
613
  self.response_model: str | None = None
498
614
  self.metadata_truncated = False
499
- self.budget = telemetry_dev.CaptureBudget(
500
- _STREAM_CAPTURE_MAX_BYTES, _STREAM_CAPTURE_MAX_ITEMS
501
- )
615
+ self.budget = _stream_budget()
502
616
  self.block_reservations: dict[int, tuple[int, int]] = {}
503
617
  self.unresolved_replacements: set[int] = set()
504
618
  self.unresolved_encrypted_content: set[int] = set()
505
619
  self.envelope_charged = False
506
620
  self.settled_tool_blocks: set[int] = set()
507
621
 
622
+ @property
623
+ def blocks(self) -> dict[int, dict[str, Any]]:
624
+ self._join_appended()
625
+ return self._blocks
626
+
627
+ @property
628
+ def tool_json(self) -> dict[int, str]:
629
+ self._join_appended()
630
+ return self._tool_json
508
631
 
509
- _STREAM_CAPTURE_MAX_BYTES = 48 * 1024
632
+ def _join_appended(self) -> None:
633
+ for (index, key), parts in self._appended_text.items():
634
+ block = self._blocks[index]
635
+ block[key] = block[key] + parts.getvalue()
636
+ self._appended_text.clear()
637
+ for index, parts in self._appended_json.items():
638
+ self._tool_json[index] = self._tool_json[index] + parts.getvalue()
639
+ self._appended_json.clear()
640
+
641
+
642
+ # The usage iterations a stream keeps.
510
643
  _STREAM_CAPTURE_MAX_ITEMS = 1000
511
644
  _STREAM_CAPTURE_MAX_DEPTH = 32
512
645
  _MAX_UNRESOLVED_REPLACEMENTS = _STREAM_CAPTURE_MAX_ITEMS
@@ -547,14 +680,40 @@ def _reserve_appended_text(index: int, value: Any, state: _StreamState) -> bool:
547
680
  if size > budget.remaining_bytes:
548
681
  budget.truncated = True
549
682
  return False
550
- budget.bytes_used += size
683
+ if not _reserve_bytes(budget, budget.bytes_used + size):
684
+ budget.truncated = True
685
+ return False
551
686
  held_bytes, held_items = state.block_reservations.get(index, (0, 0))
552
687
  state.block_reservations[index] = (held_bytes + size, held_items)
553
688
  return True
554
689
 
555
690
 
556
- def _bounded_native(value: Any) -> Any:
557
- remaining_items = [_STREAM_CAPTURE_MAX_ITEMS]
691
+ # An appended delta is charged only its text bytes, which may be none: the delta, its type
692
+ # and text fields are not charged.
693
+ _APPENDED_DELTA_ITEMS: Final = 5
694
+
695
+
696
+ def _event_items_bound(
697
+ index: int, state: _StreamState, *, replacing: bool = False, uncharged_items: int = 0
698
+ ) -> int:
699
+ """The items an event value for block ``index`` may have and still fit in the stream.
700
+
701
+ That is the bytes the stream may still keep, plus the bytes a replacement gives back,
702
+ plus the items an appended delta keeps uncharged: every other item the copy keeps is
703
+ charged at least a byte, 16 by CaptureBudget.accept and one in compact JSON. The copy
704
+ also reads two items for each None field it drops; the SDK's appended deltas have none,
705
+ and elsewhere each kept item's 16 bytes leave room for seven.
706
+ """
707
+ room = max(0, state.budget.remaining_bytes)
708
+ if replacing:
709
+ room += state.block_reservations.get(index, (0, 0))[0]
710
+ return room + uncharged_items
711
+
712
+
713
+ def _bounded_native(value: Any, max_items: int) -> Any:
714
+ """Copy an event value, or _OMIT once it passes max_items items or the depth bound, so a
715
+ value whose len() under-reports or that never ends is read only that far."""
716
+ remaining_items = [max_items]
558
717
  try:
559
718
  return _bounded_stream_native(value, remaining_items, 0, set())
560
719
  except _StreamCaptureLimit:
@@ -713,27 +872,49 @@ def _stream_output(state: _StreamState) -> list[dict[str, Any]] | None:
713
872
  ]
714
873
 
715
874
 
875
+ def _json_cost(value: Any) -> tuple[int, int] | None:
876
+ """Compact JSON bytes and items of a retained value, or None if it does not serialize.
877
+
878
+ The final bounded check and the settled tool input charge both measure with this, so
879
+ they agree on whether a settled tool block fits. Items count as CaptureBudget counts
880
+ them: each value, and each object key.
881
+ """
882
+ try:
883
+ size = len(json_dumps(value).encode())
884
+ except (TypeError, ValueError, RecursionError):
885
+ return None
886
+ # json_dumps rejects cycles, so the walk ends. It writes other values as one string.
887
+ items = 0
888
+ pending: list[Any] = [value]
889
+ while pending:
890
+ item = pending.pop()
891
+ items += 1
892
+ if isinstance(item, dict):
893
+ mapping = cast(dict[Any, Any], item)
894
+ items += len(mapping)
895
+ pending.extend(mapping.values())
896
+ elif isinstance(item, list | tuple):
897
+ pending.extend(cast(Sequence[Any], item))
898
+ return size, items
899
+
900
+
716
901
  def _bounded_stream_output(
717
902
  blocks: list[dict[str, Any]], max_bytes: int, max_items: int
718
903
  ) -> tuple[list[dict[str, Any]] | None, bool]:
719
- def encoded_size(value: Any) -> int:
720
- return len(json_dumps(value).encode())
721
-
722
- envelope: list[dict[str, Any]] = [{"role": "assistant", "content": []}]
723
- item_budget = telemetry_dev.CaptureBudget(max_items=max_items)
724
904
  kept: list[dict[str, Any]] = []
725
- try:
726
- size = encoded_size(envelope)
727
- if not item_budget.accept(envelope):
728
- return None, True
729
- for block in blocks:
730
- block_size = encoded_size(block) + (_BLOCK_SEPARATOR_BYTES if kept else 0)
731
- if size + block_size > max_bytes or not item_budget.accept(block):
732
- break
733
- size += block_size
734
- kept.append(block)
735
- except (TypeError, ValueError, RecursionError):
905
+ size, items = _ENVELOPE_BYTES, _ENVELOPE_ITEMS
906
+ if items > max_items:
736
907
  return None, True
908
+ for block in blocks:
909
+ cost = _json_cost(block)
910
+ if cost is None:
911
+ return None, True
912
+ block_size = cost[0] + (_BLOCK_SEPARATOR_BYTES if kept else 0)
913
+ if size + block_size > max_bytes or items + cost[1] > max_items:
914
+ break
915
+ size += block_size
916
+ items += cost[1]
917
+ kept.append(block)
737
918
  truncated = len(kept) < len(blocks)
738
919
  return ([{"role": "assistant", "content": kept}] if kept else None), truncated
739
920
 
@@ -784,17 +965,36 @@ def _stream_partial(
784
965
  return fields
785
966
 
786
967
 
787
- def _append_string(target: dict[str, Any], key: str, value: Any) -> None:
968
+ # The appending helpers write the buffers of _StreamState directly: its blocks and tool_json
969
+ # properties join the buffers, which on every delta would copy all the text before it.
970
+ def _append_string(
971
+ index: int, target: dict[str, Any], key: str, value: Any, state: _StreamState
972
+ ) -> None:
788
973
  text = _string(value)
789
974
  if text is None:
790
975
  return
791
- target[key] = f"{_string(target.get(key)) or ''}{text}"
976
+ parts = state._appended_text.get((index, key)) # pyright: ignore[reportPrivateUsage]
977
+ if parts is None:
978
+ target[key] = _string(target.get(key)) or ""
979
+ parts = state._appended_text[(index, key)] = _DeltaBuffer() # pyright: ignore[reportPrivateUsage]
980
+ parts.write(text)
981
+
982
+
983
+ def _append_tool_json(index: int, value: Any, state: _StreamState) -> None:
984
+ parts = state._appended_json.get(index) # pyright: ignore[reportPrivateUsage]
985
+ if parts is None:
986
+ state._tool_json.setdefault(index, "") # pyright: ignore[reportPrivateUsage]
987
+ parts = state._appended_json[index] = _DeltaBuffer() # pyright: ignore[reportPrivateUsage]
988
+ parts.write(_string(value) or "")
792
989
 
793
990
 
794
991
  def _append_item(target: dict[str, Any], key: str, value: Any) -> None:
795
992
  if value is None:
796
993
  return
797
994
  existing = target.get(key)
995
+ if isinstance(existing, list):
996
+ cast(list[Any], existing).append(_native(value))
997
+ return
798
998
  items: list[Any] = (
799
999
  list(cast(Sequence[Any], existing))
800
1000
  if isinstance(existing, Sequence) and not isinstance(existing, str)
@@ -811,7 +1011,10 @@ def _record_content_block_start(event: Any, state: _StreamState) -> None:
811
1011
  block_index = index if isinstance(index, int) else len(state.blocks)
812
1012
  raw_content_block = _field(event, "content_block")
813
1013
  raw_block_type = _string(_field(raw_content_block, "type"))
814
- content_block = _bounded_native(raw_content_block)
1014
+ content_block = _bounded_native(
1015
+ raw_content_block,
1016
+ _event_items_bound(block_index, state, replacing=raw_block_type == "compaction"),
1017
+ )
815
1018
  if content_block is _OMIT:
816
1019
  if raw_block_type == "compaction":
817
1020
  _reject_replacement(block_index, state, recover_unseen=True)
@@ -872,12 +1075,14 @@ def _default_block(delta_type: str | None) -> dict[str, Any]:
872
1075
 
873
1076
 
874
1077
  def _block_for_delta(index: int, delta_type: str | None, state: _StreamState) -> dict[str, Any]:
875
- if index in state.blocks:
876
- return state.blocks[index]
877
- state.blocks[index] = _default_block(delta_type)
1078
+ # Called for appended deltas, so it must not join the appended text.
1079
+ blocks = state._blocks # pyright: ignore[reportPrivateUsage]
1080
+ if index in blocks:
1081
+ return blocks[index]
1082
+ blocks[index] = _default_block(delta_type)
878
1083
  if delta_type == "input_json_delta":
879
- state.tool_json[index] = ""
880
- return state.blocks[index]
1084
+ state._tool_json[index] = "" # pyright: ignore[reportPrivateUsage]
1085
+ return blocks[index]
881
1086
 
882
1087
 
883
1088
  _REPLACEMENT_DELTAS = frozenset({"compaction_delta", "signature_delta"})
@@ -939,7 +1144,9 @@ def _reserve_replacement(
939
1144
  return False
940
1145
  measured_bytes = candidate.bytes_used - base_bytes
941
1146
  measured_items = candidate.items_used - base_items
942
- budget.bytes_used = base_bytes + measured_bytes
1147
+ if not _reserve_bytes(budget, base_bytes + measured_bytes):
1148
+ _reject_replacement(index, state, recover_unseen=recover_unseen)
1149
+ return False
943
1150
  budget.items_used = base_items + measured_items
944
1151
  state.block_reservations[index] = (measured_bytes, measured_items)
945
1152
  state.unresolved_replacements.discard(index)
@@ -1007,8 +1214,7 @@ def _drop_stale_signature(index: int, state: _StreamState) -> None:
1007
1214
  candidate.items_used = base_items
1008
1215
  if candidate.accept(retained):
1009
1216
  measured = (candidate.bytes_used - base_bytes, candidate.items_used - base_items)
1010
- if measured is not None:
1011
- state.budget.bytes_used = base_bytes + measured[0]
1217
+ if measured is not None and _reserve_bytes(state.budget, base_bytes + measured[0]):
1012
1218
  state.budget.items_used = base_items + measured[1]
1013
1219
  state.block_reservations[index] = measured
1014
1220
  state.blocks[index] = stripped
@@ -1030,7 +1236,17 @@ def _record_content_block_delta(event: Any, state: _StreamState) -> None:
1030
1236
  raw_delta = _field(event, "delta")
1031
1237
  raw_delta_type = _string(_field(raw_delta, "type"))
1032
1238
  replacing = raw_delta_type in _REPLACEMENT_DELTAS
1033
- delta = _bounded_native(raw_delta)
1239
+ delta = _bounded_native(
1240
+ raw_delta,
1241
+ _event_items_bound(
1242
+ block_index,
1243
+ state,
1244
+ replacing=replacing,
1245
+ uncharged_items=_APPENDED_DELTA_ITEMS
1246
+ if raw_delta_type in _APPENDED_DELTA_FIELDS
1247
+ else 0,
1248
+ ),
1249
+ )
1034
1250
  if delta is _OMIT:
1035
1251
  if replacing:
1036
1252
  _drop_rejected_replacement(
@@ -1073,7 +1289,7 @@ def _record_content_block_delta(event: Any, state: _StreamState) -> None:
1073
1289
  return
1074
1290
  appended_field = _APPENDED_DELTA_FIELDS.get(delta_type or "")
1075
1291
  if appended_field is not None:
1076
- if block_index not in state.blocks and not _reserve(
1292
+ if block_index not in state._blocks and not _reserve( # pyright: ignore[reportPrivateUsage]
1077
1293
  block_index, _default_block(delta_type), state
1078
1294
  ):
1079
1295
  return
@@ -1083,15 +1299,13 @@ def _record_content_block_delta(event: Any, state: _StreamState) -> None:
1083
1299
  return
1084
1300
  block = _block_for_delta(block_index, delta_type, state)
1085
1301
  if delta_type == "input_json_delta":
1086
- state.tool_json[block_index] = (
1087
- f"{state.tool_json.get(block_index, '')}{_string(_field(delta, 'partial_json')) or ''}"
1088
- )
1302
+ _append_tool_json(block_index, _field(delta, "partial_json"), state)
1089
1303
  elif delta_type == "text_delta":
1090
- _append_string(block, "text", _field(delta, "text"))
1304
+ _append_string(block_index, block, "text", _field(delta, "text"), state)
1091
1305
  elif delta_type == "citations_delta":
1092
1306
  _append_item(block, "citations", _field(delta, "citation"))
1093
1307
  elif delta_type == "thinking_delta":
1094
- _append_string(block, "thinking", _field(delta, "thinking"))
1308
+ _append_string(block_index, block, "thinking", _field(delta, "thinking"), state)
1095
1309
 
1096
1310
 
1097
1311
  def _fallback_serving_model(usage: Any, state: _StreamState) -> str | None:
@@ -1156,30 +1370,22 @@ _STREAM_OUTPUT_ENVELOPE: Final[list[dict[str, Any]]] = [{"role": "assistant", "c
1156
1370
  _BLOCK_SEPARATOR_BYTES: Final = 1
1157
1371
 
1158
1372
 
1159
- def _envelope_cost() -> tuple[int, int]:
1160
- measured = telemetry_dev.CaptureBudget()
1161
- measured.accept(_STREAM_OUTPUT_ENVELOPE)
1162
- envelope_bytes = len(json_dumps(_STREAM_OUTPUT_ENVELOPE).encode())
1163
- return envelope_bytes, measured.items_used
1164
-
1165
-
1166
- _ENVELOPE_BYTES, _ENVELOPE_ITEMS = _envelope_cost()
1373
+ _ENVELOPE_COST = _json_cost(_STREAM_OUTPUT_ENVELOPE)
1374
+ assert _ENVELOPE_COST is not None
1375
+ _ENVELOPE_BYTES, _ENVELOPE_ITEMS = _ENVELOPE_COST
1167
1376
 
1168
1377
 
1169
1378
  def _settled_cost(
1170
1379
  index: int, block: Mapping[str, Any], state: _StreamState
1171
1380
  ) -> tuple[int, int] | None:
1172
1381
  """Bytes and items the final bounded check counts for a block, or None if it cannot."""
1173
- try:
1174
- size = len(json_dumps(block).encode())
1175
- except (TypeError, ValueError, RecursionError):
1382
+ cost = _json_cost(block)
1383
+ if cost is None:
1176
1384
  return None
1385
+ size, items = cost
1177
1386
  if any(other < index for other in state.blocks):
1178
1387
  size += _BLOCK_SEPARATOR_BYTES
1179
- measured = telemetry_dev.CaptureBudget(state.budget.max_bytes, state.budget.max_items)
1180
- if not measured.accept(block):
1181
- return None
1182
- return size, measured.items_used
1388
+ return size, items
1183
1389
 
1184
1390
 
1185
1391
  def _settled_replacement_cost(
@@ -1225,7 +1431,9 @@ def _settle_tool_input(index: int, state: _StreamState) -> None:
1225
1431
  if base_bytes + size > budget.max_bytes or base_items + items > budget.max_items:
1226
1432
  budget.truncated = True
1227
1433
  return
1228
- budget.bytes_used = base_bytes + size
1434
+ if not _reserve_bytes(budget, base_bytes + size):
1435
+ budget.truncated = True
1436
+ return
1229
1437
  budget.items_used = base_items + items
1230
1438
  state.envelope_charged = True
1231
1439
  state.block_reservations[index] = (size, items)
@@ -1307,12 +1515,16 @@ class _InstrumentedStream:
1307
1515
  )
1308
1516
 
1309
1517
  def finish(self, error: BaseException | None = None) -> None:
1310
- fields = _stream_partial(
1311
- self._state, mask_output_when_incomplete=self._mask_output_when_incomplete
1312
- )
1313
- if error is not None:
1314
- fields["error"] = error
1315
- self._end(**fields)
1518
+ # The output is read before the release, which fixes the stream's bound.
1519
+ try:
1520
+ fields = _stream_partial(
1521
+ self._state, mask_output_when_incomplete=self._mask_output_when_incomplete
1522
+ )
1523
+ if error is not None:
1524
+ fields["error"] = error
1525
+ self._end(**fields)
1526
+ finally:
1527
+ _release_budget(self._state.budget)
1316
1528
 
1317
1529
  def _iterate(self) -> Iterator[Any]:
1318
1530
  try:
@@ -1394,12 +1606,16 @@ class _InstrumentedAsyncStream:
1394
1606
  )
1395
1607
 
1396
1608
  def finish(self, error: BaseException | None = None) -> None:
1397
- fields = _stream_partial(
1398
- self._state, mask_output_when_incomplete=self._mask_output_when_incomplete
1399
- )
1400
- if error is not None:
1401
- fields["error"] = error
1402
- self._end(**fields)
1609
+ # The output is read before the release, which fixes the stream's bound.
1610
+ try:
1611
+ fields = _stream_partial(
1612
+ self._state, mask_output_when_incomplete=self._mask_output_when_incomplete
1613
+ )
1614
+ if error is not None:
1615
+ fields["error"] = error
1616
+ self._end(**fields)
1617
+ finally:
1618
+ _release_budget(self._state.budget)
1403
1619
 
1404
1620
  async def _aiterate(self) -> AsyncIterator[Any]:
1405
1621
  try: