vantage-litellm-callback 0.0.3__tar.gz

vantage_litellm_callback-0.0.3/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.4
+Name: vantage-litellm-callback
+Version: 0.0.3
+Summary: Durable Vantage lifecycle callback for LiteLLM
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: litellm
+Requires-Dist: pydantic>=2
+
+# Vantage LiteLLM Callback
+
+Install a released version with `pip install vantage-litellm-callback`. To work
+from a source checkout, install this directory with `pip install .`. Configure
+LiteLLM with:
+
+```yaml
+litellm_settings:
+  callbacks:
+    - vantage_callback.callback_instance
+```
+
+See the repository README for collector setup and lifecycle billing semantics.
+
+The callback is fail-open: collector outages, acknowledgement failures, malformed
+responses, queue pressure, and projection errors never block or fail a provider
+request. Usage events are placed into a bounded in-process queue and delivered by
+a background worker. When delivery is unhealthy, the worker opens a circuit,
+aggregates bounded loss information, and periodically probes the collector with a
+versioned `delivery_gap` control frame. Normal delivery resumes only after that
+frame receives a durable acknowledgement.
+
+Configuration:
+
+- `VANTAGE_COLLECTOR_DELIVERY_QUEUE_SIZE` (default `1024`)
+- `VANTAGE_COLLECTOR_RECOVERY_PROBE_SECONDS` (default `5`)
+- `VANTAGE_COLLECTOR_GAP_EVENT_ID_SAMPLE_SIZE` (default `32`)
+- `VANTAGE_COLLECTOR_GAP_SUMMARY_INTERVAL_SECONDS` (default `60`)
+- `VANTAGE_COLLECTOR_SOCKET_PATH` (default
+  `/var/run/vantage-collector/collector.sock`)
+- `VANTAGE_COLLECTOR_ACK_TIMEOUT_SECONDS` (default `5`)
+- `VANTAGE_COLLECTOR_CONNECTION_POOL_SIZE` (default `16`)
+
+The queue and gap aggregate are intentionally process-local and bounded. A
+process exit may therefore lose queued events; the gap frame reports losses
+observed while the process remains alive.
+
+## Running the provider matrix locally
+
+`tests/test_provider_matrix.py` is a regression matrix over the provider and
+mode combinations whose usage shapes disagree with one another. Every cell is a
+recorded fixture, so the matrix needs no API keys and runs on every pull
+request:
+
+```sh
+pip install --editable . pytest pytest-asyncio
+PYTHONPATH=. pytest tests/test_provider_matrix.py -v
+```
+
+Run a single cell while iterating on a provider:
+
+```sh
+PYTHONPATH=. pytest tests/test_provider_matrix.py -k anthropic
+```
+
+Fixtures live in `tests/fixtures/`, one JSON file per cell:
+
+| Directory | Cells |
+| --- | --- |
+| `nonstreaming/` | OpenAI chat + cache, OpenAI multimodal, Anthropic post-`calculate_usage()`, Bedrock converse, Responses API + cache (with and without a `text_tokens` detail), OCR / non-token |
+| `streaming/` | OpenAI SSE, Anthropic `/v1/messages` SSE, Bedrock converse, Responses API `response.completed` |
+
+Each file carries its `usage` payload, the `expected_usage` projection, the
+`expected_billable_total`, and a `note` citing the LiteLLM transform the shape
+was taken from. Adding a provider means adding a JSON file — the tests
+parametrize over the directory, so a new file becomes a new cell with no test
+changes.
+
+The matrix asserts three things per cell: the four projected `Usage` fields,
+that those fields are mutually exclusive (they must sum to the billable total,
+so no token is counted twice), and that the callback emits exactly one `started`
+and one terminal event. Two further tests assert that nothing the callback
+attaches to a request reaches the outbound provider body.
+
+Two token-semantics rules are what the matrix exists to protect, both of which
+have regressed before:
+
+- **The input count is gross wherever the cache is reported nested.** LiteLLM's
+  transforms fold cache reads and writes into `prompt_tokens`, and the Responses
+  API counts `input_tokens_details.cached_tokens` inside `input_tokens`. Those
+  buckets have to be subtracted or the cached tokens bill twice. Raw Anthropic
+  usage is the exception: it puts `cache_*_input_tokens` beside a *net*
+  `input_tokens`, so nothing is subtracted there.
+- **`text_tokens` is not "uncached input".** It means the text modality for
+  OpenAI but the net raw input for Anthropic. Reading it verbatim silently drops
+  image, video, and audio input tokens, so uncached input is derived from the
+  gross count instead.
+
+For an end-to-end check against a live proxy on `:4000`, `demo/send_requests.sh`
+sends tagged requests through LiteLLM; see `demo/README.md`. That path needs a
+running proxy and collector and is a manual smoke test, not part of CI.

vantage_litellm_callback-0.0.3/README.md

@@ -0,0 +1,91 @@
+# Vantage LiteLLM Callback
+
+Install a released version with `pip install vantage-litellm-callback`. To work
+from a source checkout, install this directory with `pip install .`. Configure
+LiteLLM with:
+
+```yaml
+litellm_settings:
+  callbacks:
+    - vantage_callback.callback_instance
+```
+
+See the repository README for collector setup and lifecycle billing semantics.
+
+The callback is fail-open: collector outages, acknowledgement failures, malformed
+responses, queue pressure, and projection errors never block or fail a provider
+request. Usage events are placed into a bounded in-process queue and delivered by
+a background worker. When delivery is unhealthy, the worker opens a circuit,
+aggregates bounded loss information, and periodically probes the collector with a
+versioned `delivery_gap` control frame. Normal delivery resumes only after that
+frame receives a durable acknowledgement.
+
+Configuration:
+
+- `VANTAGE_COLLECTOR_DELIVERY_QUEUE_SIZE` (default `1024`)
+- `VANTAGE_COLLECTOR_RECOVERY_PROBE_SECONDS` (default `5`)
+- `VANTAGE_COLLECTOR_GAP_EVENT_ID_SAMPLE_SIZE` (default `32`)
+- `VANTAGE_COLLECTOR_GAP_SUMMARY_INTERVAL_SECONDS` (default `60`)
+- `VANTAGE_COLLECTOR_SOCKET_PATH` (default
+  `/var/run/vantage-collector/collector.sock`)
+- `VANTAGE_COLLECTOR_ACK_TIMEOUT_SECONDS` (default `5`)
+- `VANTAGE_COLLECTOR_CONNECTION_POOL_SIZE` (default `16`)
+
+The queue and gap aggregate are intentionally process-local and bounded. A
+process exit may therefore lose queued events; the gap frame reports losses
+observed while the process remains alive.
+
+## Running the provider matrix locally
+
+`tests/test_provider_matrix.py` is a regression matrix over the provider and
+mode combinations whose usage shapes disagree with one another. Every cell is a
+recorded fixture, so the matrix needs no API keys and runs on every pull
+request:
+
+```sh
+pip install --editable . pytest pytest-asyncio
+PYTHONPATH=. pytest tests/test_provider_matrix.py -v
+```
+
+Run a single cell while iterating on a provider:
+
+```sh
+PYTHONPATH=. pytest tests/test_provider_matrix.py -k anthropic
+```
+
+Fixtures live in `tests/fixtures/`, one JSON file per cell:
+
+| Directory | Cells |
+| --- | --- |
+| `nonstreaming/` | OpenAI chat + cache, OpenAI multimodal, Anthropic post-`calculate_usage()`, Bedrock converse, Responses API + cache (with and without a `text_tokens` detail), OCR / non-token |
+| `streaming/` | OpenAI SSE, Anthropic `/v1/messages` SSE, Bedrock converse, Responses API `response.completed` |
+
+Each file carries its `usage` payload, the `expected_usage` projection, the
+`expected_billable_total`, and a `note` citing the LiteLLM transform the shape
+was taken from. Adding a provider means adding a JSON file — the tests
+parametrize over the directory, so a new file becomes a new cell with no test
+changes.
+
+The matrix asserts three things per cell: the four projected `Usage` fields,
+that those fields are mutually exclusive (they must sum to the billable total,
+so no token is counted twice), and that the callback emits exactly one `started`
+and one terminal event. Two further tests assert that nothing the callback
+attaches to a request reaches the outbound provider body.
+
+Two token-semantics rules are what the matrix exists to protect, both of which
+have regressed before:
+
+- **The input count is gross wherever the cache is reported nested.** LiteLLM's
+  transforms fold cache reads and writes into `prompt_tokens`, and the Responses
+  API counts `input_tokens_details.cached_tokens` inside `input_tokens`. Those
+  buckets have to be subtracted or the cached tokens bill twice. Raw Anthropic
+  usage is the exception: it puts `cache_*_input_tokens` beside a *net*
+  `input_tokens`, so nothing is subtracted there.
+- **`text_tokens` is not "uncached input".** It means the text modality for
+  OpenAI but the net raw input for Anthropic. Reading it verbatim silently drops
+  image, video, and audio input tokens, so uncached input is derived from the
+  gross count instead.
+
+For an end-to-end check against a live proxy on `:4000`, `demo/send_requests.sh`
+sends tagged requests through LiteLLM; see `demo/README.md`. That path needs a
+running proxy and collector and is a manual smoke test, not part of CI.

vantage_litellm_callback-0.0.3/pyproject.toml

@@ -0,0 +1,25 @@
+[build-system]
+requires = ["setuptools>=68", "setuptools-scm[simple]>=8"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "vantage-litellm-callback"
+dynamic = ["version"]
+description = "Durable Vantage lifecycle callback for LiteLLM"
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+  "litellm",
+  "pydantic>=2",
+]
+
+[tool.setuptools]
+packages = ["vantage_litellm_callback"]
+py-modules = ["vantage_callback"]
+
+[tool.setuptools_scm]
+root = ".."
+local_scheme = "no-local-version"
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"

vantage_litellm_callback-0.0.3/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

vantage_litellm_callback-0.0.3/tests/fixtures/metadata_tags.json

@@ -0,0 +1,52 @@
+{
+  "request_data": {
+    "model": "logical-model",
+    "litellm_call_id": "request-metadata-tags",
+    "litellm_params": {
+      "model": "openai/gpt-4.1",
+      "custom_llm_provider": "openai",
+      "metadata": {
+        "tags": {
+          "team": "search",
+          "collision": "explicit-tag",
+          "enabled": true,
+          "weight": 1.5,
+          "nested_tag": {
+            "drop": "this"
+          }
+        },
+        "spend_logs_metadata": {
+          "environment": "production",
+          "collision": "spend-log",
+          "retries": 2,
+          "nested_spend_metadata": [
+            "drop",
+            "this"
+          ]
+        }
+      }
+    }
+  },
+  "collector_event": {
+    "event_id": "event-metadata-tags",
+    "request_id": "request-metadata-tags",
+    "timestamp": "2026-09-01T17:30:00Z",
+    "provider": "openai",
+    "model": "gpt-4.1",
+    "status": "success",
+    "usage": {
+      "non_cache_input_tokens": 0,
+      "output_tokens": 0,
+      "cache_read_input_tokens": 0,
+      "cache_write_input_tokens": 0
+    },
+    "tags": {
+      "team": "search",
+      "collision": "explicit-tag",
+      "enabled": true,
+      "weight": 1.5,
+      "environment": "production",
+      "retries": 2
+    }
+  }
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/anthropic_calculate_usage.json

@@ -0,0 +1,25 @@
+{
+  "provider": "anthropic",
+  "mode": "non-stream + cache",
+  "note": "Post-calculate_usage() Usage. LiteLLM inflates prompt_tokens to gross (raw input + cache read + cache creation); prompt_tokens_details.text_tokens is the net raw input. See litellm/llms/anthropic/chat/transformation.py:2189-2223.",
+  "usage": {
+    "prompt_tokens": 2000,
+    "completion_tokens": 150,
+    "total_tokens": 2150,
+    "prompt_tokens_details": {
+      "cached_tokens": 1500,
+      "text_tokens": 200,
+      "cache_write_tokens": 300,
+      "cache_creation_tokens": 300
+    },
+    "cache_creation_input_tokens": 300,
+    "cache_read_input_tokens": 1500
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 200,
+    "output_tokens": 150,
+    "cache_read_input_tokens": 1500,
+    "cache_write_input_tokens": 300
+  },
+  "expected_billable_total": 2150
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/bedrock_converse.json

@@ -0,0 +1,25 @@
+{
+  "provider": "bedrock",
+  "mode": "converse",
+  "note": "Post-transform Bedrock converse usage. Bedrock passes totalTokens through unchanged while prompt_tokens is inflated, so total_tokens (350) intentionally disagrees with prompt+completion. See litellm/llms/bedrock/chat/converse_transformation.py:1780-1790.",
+  "usage": {
+    "prompt_tokens": 2000,
+    "completion_tokens": 150,
+    "total_tokens": 350,
+    "prompt_tokens_details": {
+      "cached_tokens": 1500,
+      "text_tokens": 200,
+      "cache_write_tokens": 300,
+      "cache_creation_tokens": 300
+    },
+    "cache_creation_input_tokens": 300,
+    "cache_read_input_tokens": 1500
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 200,
+    "output_tokens": 150,
+    "cache_read_input_tokens": 1500,
+    "cache_write_input_tokens": 300
+  },
+  "expected_billable_total": 2150
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/ocr_non_token.json

@@ -0,0 +1,20 @@
+{
+  "provider": "mistral",
+  "mode": "ocr / non-token",
+  "note": "OCR spend-tracking usage carries page meters with zeroed token counts. The collector bills tokens only, so this must project to zeros rather than inventing usage. See litellm/proxy/spend_tracking/spend_tracking_utils.py:164.",
+  "usage": {
+    "prompt_tokens": 0,
+    "completion_tokens": 0,
+    "total_tokens": 0,
+    "pages_processed": 7,
+    "credits": null,
+    "doc_size_bytes": 284729
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 0,
+    "output_tokens": 0,
+    "cache_read_input_tokens": 0,
+    "cache_write_input_tokens": 0
+  },
+  "expected_billable_total": 0
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/openai_chat_cache.json

@@ -0,0 +1,19 @@
+{
+  "provider": "openai",
+  "mode": "non-stream + cache",
+  "note": "OpenAI chat completions. prompt_tokens is gross; cached_tokens is a subset of it.",
+  "usage": {
+    "prompt_tokens": 100,
+    "completion_tokens": 20,
+    "total_tokens": 120,
+    "prompt_tokens_details": {"cached_tokens": 30},
+    "cache_creation_input_tokens": 10
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 60,
+    "output_tokens": 20,
+    "cache_read_input_tokens": 30,
+    "cache_write_input_tokens": 10
+  },
+  "expected_billable_total": 120
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/openai_multimodal.json

@@ -0,0 +1,18 @@
+{
+  "provider": "openai",
+  "mode": "non-stream multimodal",
+  "note": "Image input. prompt_tokens_details.text_tokens covers only the text modality, so billable input must include image_tokens rather than trusting text_tokens alone.",
+  "usage": {
+    "prompt_tokens": 100,
+    "completion_tokens": 20,
+    "total_tokens": 120,
+    "prompt_tokens_details": {"text_tokens": 40, "image_tokens": 60}
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 100,
+    "output_tokens": 20,
+    "cache_read_input_tokens": 0,
+    "cache_write_input_tokens": 0
+  },
+  "expected_billable_total": 120
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/openai_multimodal_cache.json

@@ -0,0 +1,18 @@
+{
+  "provider": "openai",
+  "mode": "non-stream multimodal + cache",
+  "note": "Image input on a cached request. LiteLLM bills the prompt detail buckets additively (text + cache_hit + audio + image + video, see litellm/litellm_core_utils/llm_cost_calc/utils.py:652-676), so text_tokens is already net of cached_tokens and the modality counts must not have the cache subtracted from them again.",
+  "usage": {
+    "prompt_tokens": 100,
+    "completion_tokens": 20,
+    "total_tokens": 120,
+    "prompt_tokens_details": {"text_tokens": 40, "image_tokens": 30, "cached_tokens": 30}
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 70,
+    "output_tokens": 20,
+    "cache_read_input_tokens": 30,
+    "cache_write_input_tokens": 0
+  },
+  "expected_billable_total": 120
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/openai_zero_modality_cache.json

@@ -0,0 +1,19 @@
+{
+  "provider": "openai",
+  "mode": "non-stream text-only, zero modality detail",
+  "note": "A text-only cached request whose detail block still carries image_tokens: 0. Treating any present modality key as a multimodal breakdown projects the input as zero and under-bills by the whole uncached remainder, so only a positive modality count may select that path.",
+  "usage": {
+    "prompt_tokens": 100,
+    "completion_tokens": 20,
+    "total_tokens": 120,
+    "prompt_tokens_details": {"image_tokens": 0, "cached_tokens": 30},
+    "cache_creation_input_tokens": 10
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 60,
+    "output_tokens": 20,
+    "cache_read_input_tokens": 30,
+    "cache_write_input_tokens": 10
+  },
+  "expected_billable_total": 120
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/responses_api_cache.json

@@ -0,0 +1,19 @@
+{
+  "provider": "openai",
+  "mode": "responses api + cache",
+  "note": "Responses API usage. input_tokens is gross and cached_tokens is a subset of it, so cache must be subtracted or the same tokens bill twice. See litellm/types/llms/openai.py:1173-1186.",
+  "usage": {
+    "input_tokens": 2000,
+    "input_tokens_details": {"cached_tokens": 1792, "text_tokens": 208},
+    "output_tokens": 350,
+    "output_tokens_details": {"reasoning_tokens": 256, "text_tokens": 94},
+    "total_tokens": 2350
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 208,
+    "output_tokens": 350,
+    "cache_read_input_tokens": 1792,
+    "cache_write_input_tokens": 0
+  },
+  "expected_billable_total": 2350
+}

vantage_litellm_callback-0.0.3/tests/fixtures/nonstreaming/responses_api_cache_no_text_detail.json

@@ -0,0 +1,18 @@
+{
+  "provider": "openai",
+  "mode": "responses api + cache, no text detail",
+  "note": "InputTokensDetails.text_tokens is optional while cached_tokens defaults to 0, so a cached Responses API call can report only the cache subset. input_tokens stays gross, so the cache must still be subtracted; counting it on both sides billed 4142 tokens for this 2350-token request. See litellm/types/llms/openai.py:1173.",
+  "usage": {
+    "input_tokens": 2000,
+    "input_tokens_details": {"cached_tokens": 1792},
+    "output_tokens": 350,
+    "total_tokens": 2350
+  },
+  "expected_usage": {
+    "non_cache_input_tokens": 208,
+    "output_tokens": 350,
+    "cache_read_input_tokens": 1792,
+    "cache_write_input_tokens": 0
+  },
+  "expected_billable_total": 2350
+}

vantage_litellm_callback-0.0.3/tests/fixtures/streaming/anthropic_messages.json

@@ -0,0 +1,16 @@
+{
+  "chunks": [
+    "event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"msg_01XYZ\",\"model\":\"claude-sonnet-4-20250514\",\"usage\":{\"input_tokens\":60,\"cache_read_input_tokens\":30,\"cache_creation_input_tokens\":10,\"output_tokens\":2}}}\n\n",
+    "event: content_block_start\ndata: {\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n",
+    "event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Hello\"}}\n\n",
+    "event: content_block_stop\ndata: {\"type\":\"content_block_stop\",\"index\":0}\n\n",
+    "event: message_delta\ndata: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\",\"stop_sequence\":null},\"usage\":{\"output_tokens\":20}}\n\n",
+    "event: message_stop\ndata: {\"type\":\"message_stop\"}\n\n"
+  ],
+  "expected_usage": {
+    "non_cache_input_tokens": 60,
+    "output_tokens": 20,
+    "cache_read_input_tokens": 30,
+    "cache_write_input_tokens": 10
+  }
+}

vantage_litellm_callback-0.0.3/tests/fixtures/streaming/bedrock_converse.json

@@ -0,0 +1,32 @@
+{
+  "chunks": [
+    {
+      "id": "chatcmpl-bedrock-1",
+      "model": "anthropic.claude-sonnet-4-20250514-v1:0",
+      "choices": [{"delta": {"content": "Hello"}, "finish_reason": null}]
+    },
+    {
+      "id": "chatcmpl-bedrock-1",
+      "model": "anthropic.claude-sonnet-4-20250514-v1:0",
+      "choices": [{"delta": {}, "finish_reason": "stop"}],
+      "usage": {
+        "prompt_tokens": 100,
+        "completion_tokens": 20,
+        "total_tokens": 120,
+        "prompt_tokens_details": {
+          "text_tokens": 60,
+          "cached_tokens": 30,
+          "cache_creation_tokens": 10
+        },
+        "cache_read_input_tokens": 30,
+        "cache_creation_input_tokens": 10
+      }
+    }
+  ],
+  "expected_usage": {
+    "non_cache_input_tokens": 60,
+    "output_tokens": 20,
+    "cache_read_input_tokens": 30,
+    "cache_write_input_tokens": 10
+  }
+}

vantage_litellm_callback-0.0.3/tests/fixtures/streaming/openai_chat.json

@@ -0,0 +1,32 @@
+{
+  "chunks": [
+    {
+      "id": "chatcmpl-oai-1",
+      "model": "gpt-4.1",
+      "choices": [{"delta": {"content": "Hello"}, "finish_reason": null}]
+    },
+    {
+      "id": "chatcmpl-oai-1",
+      "model": "gpt-4.1",
+      "choices": [{"delta": {"content": " world"}, "finish_reason": null}]
+    },
+    {
+      "id": "chatcmpl-oai-1",
+      "model": "gpt-4.1",
+      "choices": [{"delta": {}, "finish_reason": "stop"}],
+      "usage": {
+        "prompt_tokens": 100,
+        "completion_tokens": 20,
+        "total_tokens": 120,
+        "prompt_tokens_details": {"cached_tokens": 30},
+        "cache_creation_input_tokens": 10
+      }
+    }
+  ],
+  "expected_usage": {
+    "non_cache_input_tokens": 60,
+    "output_tokens": 20,
+    "cache_read_input_tokens": 30,
+    "cache_write_input_tokens": 10
+  }
+}

vantage_litellm_callback-0.0.3/tests/fixtures/streaming/responses_api.json

@@ -0,0 +1,24 @@
+{
+  "chunks": [
+    {
+      "type": "response.created",
+      "response": {"id": "resp_1", "model": "gpt-4.1", "status": "in_progress"}
+    },
+    {"type": "response.output_text.delta", "delta": "Hello"},
+    {
+      "type": "response.completed",
+      "response": {
+        "id": "resp_1",
+        "model": "gpt-4.1",
+        "status": "completed",
+        "usage": {"input_tokens": 9, "output_tokens": 4, "total_tokens": 13}
+      }
+    }
+  ],
+  "expected_usage": {
+    "non_cache_input_tokens": 9,
+    "output_tokens": 4,
+    "cache_read_input_tokens": 0,
+    "cache_write_input_tokens": 0
+  }
+}