tokenmeter-sdk 0.2.0__tar.gz → 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {tokenmeter_sdk-0.2.0/src/tokenmeter_sdk.egg-info → tokenmeter_sdk-0.2.1}/PKG-INFO +33 -15
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/README.md +32 -14
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/pyproject.toml +1 -1
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/__init__.py +3 -2
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_buffer.py +143 -16
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_diagnose.py +2 -2
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_preflight.py +42 -4
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_topic.py +91 -91
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_transport.py +1 -1
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1/src/tokenmeter_sdk.egg-info}/PKG-INFO +33 -15
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/LICENSE +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/MANIFEST.in +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/setup.cfg +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_client.py +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_context.py +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_ids.py +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_normalize.py +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_optimization.py +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_read_api.py +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter/_wrap.py +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter_sdk.egg-info/SOURCES.txt +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter_sdk.egg-info/dependency_links.txt +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter_sdk.egg-info/requires.txt +0 -0
- {tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter_sdk.egg-info/top_level.txt +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: tokenmeter-sdk
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Summary: Token Meter Python SDK: wraps OpenAI/Anthropic/Gemini/Bedrock/Azure clients and meters token usage
|
|
5
5
|
Author: Saqlain Rashid
|
|
6
6
|
License-Expression: MIT
|
|
@@ -36,20 +36,6 @@ Dynamic: license-file
|
|
|
36
36
|
|
|
37
37
|
# token-meter (Python SDK)
|
|
38
38
|
|
|
39
|
-
## Read usage data
|
|
40
|
-
|
|
41
|
-
Create a separate read-only key under **Settings → API keys → Read API keys**. It cannot submit usage events.
|
|
42
|
-
|
|
43
|
-
```python
|
|
44
|
-
import os
|
|
45
|
-
from tokenmeter import TokenMeterReadClient
|
|
46
|
-
|
|
47
|
-
read_api = TokenMeterReadClient(os.environ["TOKEN_METER_READ_KEY"], os.environ["TOKEN_METER_DASHBOARD_URL"])
|
|
48
|
-
page = read_api.list_daily_usage("2026-10-01", "2026-10-31")
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
This returns daily **metered** usage/spend summaries, not provider invoice data. See [`docs/openapi.yaml`](../../docs/openapi.yaml) for pagination, fields, and limits.
|
|
52
|
-
|
|
53
39
|
In-process LLM usage capture for Python. `tokenmeter.wrap(client)` wraps a
|
|
54
40
|
provider client you already have, reads token usage out of the response
|
|
55
41
|
object it already returns (no extra network hop), normalizes it to a
|
|
@@ -128,6 +114,38 @@ client = tokenmeter.wrap_openai_compatible(
|
|
|
128
114
|
)
|
|
129
115
|
```
|
|
130
116
|
|
|
117
|
+
## Read usage data
|
|
118
|
+
|
|
119
|
+
Create a separate read-only key under **Settings → API keys → Read API keys**. It cannot submit usage events.
|
|
120
|
+
|
|
121
|
+
```python
|
|
122
|
+
import os
|
|
123
|
+
from tokenmeter import TokenMeterReadClient
|
|
124
|
+
|
|
125
|
+
read_api = TokenMeterReadClient(os.environ["TOKEN_METER_READ_KEY"], os.environ["TOKEN_METER_DASHBOARD_URL"])
|
|
126
|
+
page = read_api.list_daily_usage("2026-10-01", "2026-10-31")
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
This returns daily **metered** usage/spend summaries, not provider invoice data. Rows use camelCase keys (`pricedCostUsd`, `nextCursor`). See the [API reference](https://timewastevc.com/docs/reference#read-api) for pagination and fields.
|
|
130
|
+
|
|
131
|
+
## Governance preflight
|
|
132
|
+
|
|
133
|
+
For calls made to a provider directly (not through the Token Meter router), ask the router's control plane first
|
|
134
|
+
whether the call is allowed:
|
|
135
|
+
|
|
136
|
+
```python
|
|
137
|
+
tokenmeter.init(ingest_key="tl_live_...", router_url="https://router.timewastevc.com")
|
|
138
|
+
|
|
139
|
+
with tokenmeter.governed(provider="openai", model="gpt-4o", content=prompt, max_output_tokens=800):
|
|
140
|
+
client.chat.completions.create(model="gpt-4o", messages=messages, max_tokens=800)
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
Unlike metering, this is on the request path: it adds one HTTP call to the router (2 s timeout), raises
|
|
144
|
+
`tokenmeter.GovernanceDenied` on a denial (`governance_mode="advisory"` only records it), and raises
|
|
145
|
+
`tokenmeter.GovernanceUnavailable` when the router can't be reached (`governance_fail_mode="open"` proceeds
|
|
146
|
+
unprotected). A wrong or revoked ingest key (401, 403) or a malformed request (400) always raises
|
|
147
|
+
`tokenmeter.GovernanceRejected`, whatever the fail mode is: it is not an outage. See the [API reference](https://timewastevc.com/docs/reference#preflight).
|
|
148
|
+
|
|
131
149
|
## Provider support matrix
|
|
132
150
|
|
|
133
151
|
| Provider | Client | Paths wrapped | Streaming | Normalization confidence |
|
|
@@ -1,19 +1,5 @@
|
|
|
1
1
|
# token-meter (Python SDK)
|
|
2
2
|
|
|
3
|
-
## Read usage data
|
|
4
|
-
|
|
5
|
-
Create a separate read-only key under **Settings → API keys → Read API keys**. It cannot submit usage events.
|
|
6
|
-
|
|
7
|
-
```python
|
|
8
|
-
import os
|
|
9
|
-
from tokenmeter import TokenMeterReadClient
|
|
10
|
-
|
|
11
|
-
read_api = TokenMeterReadClient(os.environ["TOKEN_METER_READ_KEY"], os.environ["TOKEN_METER_DASHBOARD_URL"])
|
|
12
|
-
page = read_api.list_daily_usage("2026-10-01", "2026-10-31")
|
|
13
|
-
```
|
|
14
|
-
|
|
15
|
-
This returns daily **metered** usage/spend summaries, not provider invoice data. See [`docs/openapi.yaml`](../../docs/openapi.yaml) for pagination, fields, and limits.
|
|
16
|
-
|
|
17
3
|
In-process LLM usage capture for Python. `tokenmeter.wrap(client)` wraps a
|
|
18
4
|
provider client you already have, reads token usage out of the response
|
|
19
5
|
object it already returns (no extra network hop), normalizes it to a
|
|
@@ -92,6 +78,38 @@ client = tokenmeter.wrap_openai_compatible(
|
|
|
92
78
|
)
|
|
93
79
|
```
|
|
94
80
|
|
|
81
|
+
## Read usage data
|
|
82
|
+
|
|
83
|
+
Create a separate read-only key under **Settings → API keys → Read API keys**. It cannot submit usage events.
|
|
84
|
+
|
|
85
|
+
```python
|
|
86
|
+
import os
|
|
87
|
+
from tokenmeter import TokenMeterReadClient
|
|
88
|
+
|
|
89
|
+
read_api = TokenMeterReadClient(os.environ["TOKEN_METER_READ_KEY"], os.environ["TOKEN_METER_DASHBOARD_URL"])
|
|
90
|
+
page = read_api.list_daily_usage("2026-10-01", "2026-10-31")
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
This returns daily **metered** usage/spend summaries, not provider invoice data. Rows use camelCase keys (`pricedCostUsd`, `nextCursor`). See the [API reference](https://timewastevc.com/docs/reference#read-api) for pagination and fields.
|
|
94
|
+
|
|
95
|
+
## Governance preflight
|
|
96
|
+
|
|
97
|
+
For calls made to a provider directly (not through the Token Meter router), ask the router's control plane first
|
|
98
|
+
whether the call is allowed:
|
|
99
|
+
|
|
100
|
+
```python
|
|
101
|
+
tokenmeter.init(ingest_key="tl_live_...", router_url="https://router.timewastevc.com")
|
|
102
|
+
|
|
103
|
+
with tokenmeter.governed(provider="openai", model="gpt-4o", content=prompt, max_output_tokens=800):
|
|
104
|
+
client.chat.completions.create(model="gpt-4o", messages=messages, max_tokens=800)
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Unlike metering, this is on the request path: it adds one HTTP call to the router (2 s timeout), raises
|
|
108
|
+
`tokenmeter.GovernanceDenied` on a denial (`governance_mode="advisory"` only records it), and raises
|
|
109
|
+
`tokenmeter.GovernanceUnavailable` when the router can't be reached (`governance_fail_mode="open"` proceeds
|
|
110
|
+
unprotected). A wrong or revoked ingest key (401, 403) or a malformed request (400) always raises
|
|
111
|
+
`tokenmeter.GovernanceRejected`, whatever the fail mode is: it is not an outage. See the [API reference](https://timewastevc.com/docs/reference#preflight).
|
|
112
|
+
|
|
95
113
|
## Provider support matrix
|
|
96
114
|
|
|
97
115
|
| Provider | Client | Paths wrapped | Streaming | Normalization confidence |
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
from ._client import init
|
|
11
11
|
from ._context import TraceContext, action, attach, current
|
|
12
12
|
from ._diagnose import diagnose
|
|
13
|
-
from ._preflight import GovernanceDenied, GovernanceUnavailable, PreflightDecision, governed, preflight
|
|
13
|
+
from ._preflight import GovernanceDenied, GovernanceRejected, GovernanceUnavailable, PreflightDecision, governed, preflight
|
|
14
14
|
from ._wrap import wrap, wrap_openai_compatible
|
|
15
15
|
from ._read_api import ReadApiClient, ReadApiError, TokenMeterReadClient
|
|
16
16
|
from ._topic import keyword_topic_classifier, latest_user_text
|
|
@@ -29,6 +29,7 @@ __all__ = [
|
|
|
29
29
|
"governed",
|
|
30
30
|
"PreflightDecision",
|
|
31
31
|
"GovernanceDenied",
|
|
32
|
+
"GovernanceRejected",
|
|
32
33
|
"GovernanceUnavailable",
|
|
33
34
|
"ReadApiClient",
|
|
34
35
|
"TokenMeterReadClient",
|
|
@@ -37,7 +38,7 @@ __all__ = [
|
|
|
37
38
|
"latest_user_text",
|
|
38
39
|
]
|
|
39
40
|
|
|
40
|
-
__version__ = "0.2.
|
|
41
|
+
__version__ = "0.2.1"
|
|
41
42
|
|
|
42
43
|
|
|
43
44
|
def flush(deadline_s: float = 2.0) -> None:
|
|
@@ -17,11 +17,23 @@ logger = logging.getLogger("tokenmeter")
|
|
|
17
17
|
DEFAULT_MAXSIZE = 10_000
|
|
18
18
|
DEFAULT_FLUSH_INTERVAL_S = 2.0
|
|
19
19
|
DEFAULT_FLUSH_BATCH_SIZE = 500
|
|
20
|
+
# Ingest refuses a batch with more events than this (413 batch_too_large, QUOTAS.EVENTS_PER_BATCH). The TypeScript
|
|
21
|
+
# SDK clamps to the same number (MAX_EVENTS_PER_BATCH in client.ts).
|
|
22
|
+
MAX_EVENTS_PER_BATCH = 1_000
|
|
20
23
|
# DECISIONS.md §8: sustained rate limiting must be LOUD, not a silent spin.
|
|
21
24
|
# 5 consecutive 429s at the default 2s flush interval is ~10s of sustained
|
|
22
25
|
# rejection -- past one unlucky fixed-window boundary straddle, short of
|
|
23
26
|
# letting the ~10k-event ring buffer fill silently.
|
|
24
27
|
CONSECUTIVE_429_WARN_THRESHOLD = 5
|
|
28
|
+
# After a failure that may pass (429, 5xx, network), wait before the next attempt: 1s, doubling to 60s, as the
|
|
29
|
+
# TypeScript SDK does. A 429's Retry-After is used instead when the server sends one.
|
|
30
|
+
BACKOFF_INITIAL_S = 1.0
|
|
31
|
+
BACKOFF_MAX_S = 60.0
|
|
32
|
+
RETRY_AFTER_MAX_S = 300.0
|
|
33
|
+
# Answers that will be the same however often the batch is re-sent. 401/403: the ingest key is wrong or revoked.
|
|
34
|
+
# 400: the batch itself is malformed. Retrying them is the bug this list exists to prevent.
|
|
35
|
+
_PERMANENT_STATUSES = (400, 401, 403)
|
|
36
|
+
_AUTH_STATUSES = (401, 403)
|
|
25
37
|
|
|
26
38
|
|
|
27
39
|
class RingBuffer:
|
|
@@ -80,6 +92,13 @@ class RingBuffer:
|
|
|
80
92
|
with self._lock:
|
|
81
93
|
self._dropped = max(0, self._dropped - n)
|
|
82
94
|
|
|
95
|
+
def count_dropped(self, n: int) -> None:
|
|
96
|
+
"""Add n events the flusher gave up on (a batch the server will never
|
|
97
|
+
accept) to the pending drop count, so the loss is reported with the
|
|
98
|
+
next batch that is accepted instead of disappearing."""
|
|
99
|
+
with self._lock:
|
|
100
|
+
self._dropped += n
|
|
101
|
+
|
|
83
102
|
def requeue(self, events: List[dict]) -> None:
|
|
84
103
|
"""Put a drained-but-unsent batch back, oldest-first, for a failed
|
|
85
104
|
send -- mirrors the TypeScript SDK's unshiftMany(). Without this,
|
|
@@ -99,7 +118,17 @@ class RingBuffer:
|
|
|
99
118
|
|
|
100
119
|
|
|
101
120
|
class Flusher:
|
|
102
|
-
"""Background thread: flush when len(buffer) >= batch_size OR every interval_s.
|
|
121
|
+
"""Background thread: flush when len(buffer) >= batch_size OR every interval_s.
|
|
122
|
+
|
|
123
|
+
One request carries at most `batch_size` events, never more than ingest accepts (MAX_EVENTS_PER_BATCH); a backlog
|
|
124
|
+
goes out as several requests. What happens after a failed request depends on why it failed:
|
|
125
|
+
|
|
126
|
+
- 401 / 403 / 400: the answer will not change. The batch is dropped and counted, and a rejected key is reported
|
|
127
|
+
once, loudly.
|
|
128
|
+
- 413: the batch is too large in bytes. It is put back and the next request carries half as many events; one
|
|
129
|
+
event that is too large by itself is dropped and counted.
|
|
130
|
+
- 429, 5xx, a network error: the batch is put back and the next attempt waits (Retry-After, or 1s doubling to 60s).
|
|
131
|
+
"""
|
|
103
132
|
|
|
104
133
|
def __init__(
|
|
105
134
|
self,
|
|
@@ -112,13 +141,18 @@ class Flusher:
|
|
|
112
141
|
self._buffer = buffer
|
|
113
142
|
self._send = send
|
|
114
143
|
self._interval_s = interval_s
|
|
115
|
-
self._batch_size = batch_size
|
|
144
|
+
self._batch_size = max(1, min(int(batch_size), MAX_EVENTS_PER_BATCH))
|
|
116
145
|
self._wake = threading.Event()
|
|
117
146
|
self._stop = threading.Event()
|
|
118
147
|
self._thread = threading.Thread(target=self._run, name="tokenmeter-flusher", daemon=True)
|
|
119
148
|
self._atexit_registered = False
|
|
120
149
|
self._consecutive_429s = 0
|
|
121
150
|
self._warned_sustained_429 = False
|
|
151
|
+
self._warned_rejected_key = False
|
|
152
|
+
# Set after a 413: the next request carries at most this many events. Cleared by the next accepted batch.
|
|
153
|
+
self._split_limit: Optional[int] = None
|
|
154
|
+
self._backoff_s = 0.0
|
|
155
|
+
self._retry_at = 0.0 # time.monotonic() before which the background thread does not try again
|
|
122
156
|
|
|
123
157
|
def start(self) -> None:
|
|
124
158
|
self._thread.start()
|
|
@@ -126,21 +160,40 @@ class Flusher:
|
|
|
126
160
|
atexit.register(self.flush_best_effort, deadline_s=2.0)
|
|
127
161
|
self._atexit_registered = True
|
|
128
162
|
|
|
163
|
+
def backoff_remaining_s(self) -> float:
|
|
164
|
+
"""Seconds until the background thread tries again after a failure; 0 when it is not waiting."""
|
|
165
|
+
return max(0.0, self._retry_at - time.monotonic())
|
|
166
|
+
|
|
129
167
|
def notify(self) -> None:
|
|
130
|
-
"""Called on every append; wakes the flusher early once batch_size is hit.
|
|
131
|
-
|
|
168
|
+
"""Called on every append; wakes the flusher early once batch_size is hit.
|
|
169
|
+
|
|
170
|
+
Not while it is backing off: the buffer stays full for as long as sends fail, so every emit would otherwise
|
|
171
|
+
turn the wait into an immediate retry."""
|
|
172
|
+
if len(self._buffer) >= self._batch_size and self.backoff_remaining_s() == 0:
|
|
132
173
|
self._wake.set()
|
|
133
174
|
|
|
134
175
|
def _run(self) -> None:
|
|
135
176
|
while not self._stop.is_set():
|
|
136
177
|
self._wake.wait(timeout=self._interval_s)
|
|
137
178
|
self._wake.clear()
|
|
138
|
-
|
|
179
|
+
# A wake that was already set when the failure happened must not cut the backoff short.
|
|
180
|
+
remaining = self.backoff_remaining_s()
|
|
181
|
+
if remaining > 0 and self._stop.wait(timeout=remaining):
|
|
182
|
+
break
|
|
183
|
+
self._flush_all()
|
|
184
|
+
|
|
185
|
+
def _flush_all(self) -> None:
|
|
186
|
+
"""Send batch after batch until the buffer is empty or a request fails."""
|
|
187
|
+
while self._flush_once() and len(self._buffer) > 0:
|
|
188
|
+
pass
|
|
139
189
|
|
|
140
|
-
def _flush_once(self) ->
|
|
141
|
-
|
|
190
|
+
def _flush_once(self) -> bool:
|
|
191
|
+
"""Send one batch. True when it is worth sending the next one straight away: the batch was accepted, or it
|
|
192
|
+
was refused for its size and the next one will be smaller."""
|
|
193
|
+
limit = self._batch_size if self._split_limit is None else min(self._batch_size, self._split_limit)
|
|
194
|
+
events = self._buffer.drain(limit)
|
|
142
195
|
if not events:
|
|
143
|
-
return
|
|
196
|
+
return False
|
|
144
197
|
# Peek, don't consume: the count is only "communicated" once the
|
|
145
198
|
# server actually acknowledges it. take_dropped() used to be called
|
|
146
199
|
# unconditionally here, BEFORE knowing whether send() would even
|
|
@@ -152,16 +205,69 @@ class Flusher:
|
|
|
152
205
|
try:
|
|
153
206
|
self._send(events, dropped)
|
|
154
207
|
except Exception as exc: # SDK NEVER throws into host code -- swallow, log, move on.
|
|
155
|
-
# drain() already removed these from the buffer; without putting
|
|
156
|
-
# them back, a failed send silently discarded the whole batch --
|
|
157
|
-
# not just under-counted drops, an actual data-loss bug.
|
|
158
|
-
self._buffer.requeue(events)
|
|
159
|
-
self._note_failure(exc)
|
|
160
208
|
logger.debug("tokenmeter: flush failed", exc_info=True)
|
|
161
|
-
return
|
|
209
|
+
return self._after_failure(events, exc)
|
|
162
210
|
self._buffer.release_dropped(dropped)
|
|
163
211
|
self._consecutive_429s = 0
|
|
164
212
|
self._warned_sustained_429 = False
|
|
213
|
+
self._warned_rejected_key = False
|
|
214
|
+
self._split_limit = None
|
|
215
|
+
self._backoff_s = 0.0
|
|
216
|
+
self._retry_at = 0.0
|
|
217
|
+
return True
|
|
218
|
+
|
|
219
|
+
def _after_failure(self, events: List[dict], exc: Exception) -> bool:
|
|
220
|
+
code = getattr(exc, "code", None)
|
|
221
|
+
if code in _PERMANENT_STATUSES:
|
|
222
|
+
# drain() already removed these; they are not put back, because re-sending gets the same answer.
|
|
223
|
+
self._buffer.count_dropped(len(events))
|
|
224
|
+
self._note_failure(exc)
|
|
225
|
+
if code in _AUTH_STATUSES:
|
|
226
|
+
self._warn_rejected_key(code, len(events))
|
|
227
|
+
self._back_off(exc) # the next batch gets the same answer: do not hammer the endpoint with them
|
|
228
|
+
else:
|
|
229
|
+
logger.warning("tokenmeter: ingest refused a batch of %d events as malformed (HTTP 400); they were dropped.", len(events))
|
|
230
|
+
return False
|
|
231
|
+
if code == 413:
|
|
232
|
+
self._note_failure(exc)
|
|
233
|
+
if len(events) == 1:
|
|
234
|
+
self._buffer.count_dropped(1)
|
|
235
|
+
self._split_limit = None
|
|
236
|
+
logger.warning("tokenmeter: one event is larger than ingest accepts (HTTP 413); it was dropped.")
|
|
237
|
+
return True
|
|
238
|
+
self._buffer.requeue(events)
|
|
239
|
+
self._split_limit = max(1, len(events) // 2)
|
|
240
|
+
return True
|
|
241
|
+
# 429, 5xx, a network error: drain() already removed these from the buffer; without putting
|
|
242
|
+
# them back, a failed send silently discarded the whole batch.
|
|
243
|
+
self._buffer.requeue(events)
|
|
244
|
+
self._note_failure(exc)
|
|
245
|
+
self._back_off(exc)
|
|
246
|
+
return False
|
|
247
|
+
|
|
248
|
+
def _back_off(self, exc: Exception) -> None:
|
|
249
|
+
retry_after = _retry_after_s(exc) if getattr(exc, "code", None) == 429 else None
|
|
250
|
+
if retry_after is not None:
|
|
251
|
+
wait = retry_after
|
|
252
|
+
else:
|
|
253
|
+
self._backoff_s = BACKOFF_INITIAL_S if self._backoff_s == 0 else min(self._backoff_s * 2, BACKOFF_MAX_S)
|
|
254
|
+
wait = self._backoff_s
|
|
255
|
+
self._retry_at = time.monotonic() + wait
|
|
256
|
+
|
|
257
|
+
def _warn_rejected_key(self, code: int, lost: int) -> None:
|
|
258
|
+
"""A wrong or revoked key is the host's to fix, and nothing is recorded until it is: say so once per
|
|
259
|
+
incident, where it will be seen (print + log, never raise -- as the 429 warning does)."""
|
|
260
|
+
if self._warned_rejected_key:
|
|
261
|
+
return
|
|
262
|
+
self._warned_rejected_key = True
|
|
263
|
+
reason = "is not a valid key" if code == 401 else "has been revoked or has expired"
|
|
264
|
+
msg = (
|
|
265
|
+
f"tokenmeter: ingest rejected this SDK's ingest key (HTTP {code}): it {reason}. "
|
|
266
|
+
f"{lost} events were dropped and nothing will be recorded until the key is replaced "
|
|
267
|
+
"(TOKEN_METER_KEY, or the ingest_key passed to tokenmeter.init). Run tokenmeter.diagnose() for more detail."
|
|
268
|
+
)
|
|
269
|
+
logger.warning(msg)
|
|
270
|
+
print("[tokenmeter] WARNING: " + msg)
|
|
165
271
|
|
|
166
272
|
def _note_failure(self, exc: Exception) -> None:
|
|
167
273
|
"""DECISIONS.md §8: sustained 429s must be loud, not a silent retry
|
|
@@ -188,14 +294,35 @@ class Flusher:
|
|
|
188
294
|
print("[tokenmeter] WARNING: " + msg)
|
|
189
295
|
|
|
190
296
|
def flush_best_effort(self, deadline_s: float = 2.0) -> None:
|
|
191
|
-
"""Synchronous, best-effort, capped flush -- used by atexit and public flush().
|
|
297
|
+
"""Synchronous, best-effort, capped flush -- used by atexit and public flush().
|
|
298
|
+
|
|
299
|
+
Sends batch after batch until the buffer is empty, the deadline passes, or a request fails. It stops at the
|
|
300
|
+
first failure: re-sending to an endpoint that just failed, in a tight loop until the deadline, delivers
|
|
301
|
+
nothing and holds the host's exit for the whole deadline. What is left stays buffered for the next flush."""
|
|
192
302
|
deadline = time.monotonic() + deadline_s
|
|
193
303
|
try:
|
|
194
304
|
while len(self._buffer) > 0 and time.monotonic() < deadline:
|
|
195
|
-
self._flush_once()
|
|
305
|
+
if not self._flush_once():
|
|
306
|
+
break
|
|
196
307
|
except Exception:
|
|
197
308
|
logger.debug("tokenmeter: atexit flush failed", exc_info=True)
|
|
198
309
|
|
|
199
310
|
def stop(self) -> None:
|
|
200
311
|
self._stop.set()
|
|
201
312
|
self._wake.set()
|
|
313
|
+
|
|
314
|
+
|
|
315
|
+
def _retry_after_s(exc: Exception) -> Optional[float]:
|
|
316
|
+
"""The Retry-After of a 429, in seconds, when the server sent a usable one (urllib's HTTPError carries the
|
|
317
|
+
response headers). Capped, so a wrong header cannot park the flusher for hours."""
|
|
318
|
+
headers = getattr(exc, "headers", None)
|
|
319
|
+
try:
|
|
320
|
+
raw = headers.get("Retry-After") if headers is not None else None
|
|
321
|
+
if raw is None:
|
|
322
|
+
return None
|
|
323
|
+
value = float(raw)
|
|
324
|
+
except (AttributeError, TypeError, ValueError):
|
|
325
|
+
return None
|
|
326
|
+
if value != value or value < 0: # NaN or negative
|
|
327
|
+
return None
|
|
328
|
+
return min(max(value, BACKOFF_INITIAL_S), RETRY_AFTER_MAX_S)
|
|
@@ -14,8 +14,8 @@ from ._client import DEFAULT_ENDPOINT, get_state
|
|
|
14
14
|
from ._transport import Transport
|
|
15
15
|
|
|
16
16
|
_STATUS_HINTS = {
|
|
17
|
-
401: "invalid_key: the ingest key is wrong
|
|
18
|
-
403: "key_revoked: this key
|
|
17
|
+
401: "invalid_key: the ingest key is wrong or malformed. Check tokenmeter.init(ingest_key=...).",
|
|
18
|
+
403: "key_revoked: this key was revoked or has expired, or its workspace was deleted. Issue a new key under Governance > Keys.",
|
|
19
19
|
413: "batch_too_large: unrelated to diagnose() itself -- your real traffic is sending oversized batches.",
|
|
20
20
|
429: "rate_limited: traffic looks abuse-shaped (well over the plan quota), not ordinary overage -- ordinary overage is accepted (DECISIONS.md §8). A single 429 is retried and not data loss; SUSTAINED 429s can be, since this SDK's local buffer is bounded -- watch for the sustained-429 warning this SDK logs.",
|
|
21
21
|
503: "unavailable: TokenMeter ingest is temporarily down. Events buffer locally and retry.",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
"""Governance preflight for direct provider calls (TS SDK: src/preflight.ts).
|
|
2
2
|
|
|
3
|
-
tokenmeter.init(ingest_key="tl_live_...", router_url="https://router.
|
|
3
|
+
tokenmeter.init(ingest_key="tl_live_...", router_url="https://router.timewastevc.com")
|
|
4
4
|
with tokenmeter.governed(provider="openai", model="gpt-4o", content=prompt, max_output_tokens=800):
|
|
5
5
|
client.chat.completions.create(model="gpt-4o", messages=..., max_tokens=800)
|
|
6
6
|
|
|
@@ -11,9 +11,11 @@ carries the decision id (tags.preflight_id), which is what lets TokenMeter repor
|
|
|
11
11
|
the call as preflight-protected instead of detected after the fact.
|
|
12
12
|
|
|
13
13
|
Unlike telemetry, this IS on the request path, so it is allowed to raise:
|
|
14
|
-
GovernanceDenied for a denial in "enforce" mode (default),
|
|
14
|
+
GovernanceDenied for a denial in "enforce" mode (default),
|
|
15
15
|
GovernanceUnavailable when the control plane cannot be reached and
|
|
16
|
-
fail_mode is "closed" (default)
|
|
16
|
+
fail_mode is "closed" (default), and GovernanceRejected, whatever fail_mode
|
|
17
|
+
is, when the control plane refuses the ingest key (401, 403) or the request
|
|
18
|
+
(400). stdlib only, like _transport.py.
|
|
17
19
|
"""
|
|
18
20
|
from __future__ import annotations
|
|
19
21
|
|
|
@@ -81,6 +83,37 @@ class GovernanceUnavailable(Exception):
|
|
|
81
83
|
pass
|
|
82
84
|
|
|
83
85
|
|
|
86
|
+
class GovernanceRejected(Exception):
|
|
87
|
+
"""The control plane answered, and refused the request itself: the ingest key is wrong (401) or revoked (403),
|
|
88
|
+
or the request is malformed (400). Not an outage, so fail_mode does not apply: a rejected key under
|
|
89
|
+
fail_mode="open" would otherwise let every call run unprotected for as long as the key stayed bad."""
|
|
90
|
+
|
|
91
|
+
def __init__(self, status: int, code: Optional[str]) -> None:
|
|
92
|
+
what = (
|
|
93
|
+
"the ingest key is not a valid key" if status == 401
|
|
94
|
+
else "the ingest key has been revoked or has expired" if status == 403
|
|
95
|
+
else "the preflight request is malformed"
|
|
96
|
+
)
|
|
97
|
+
super().__init__(f"TokenMeter governance preflight was rejected (HTTP {status}{' ' + code if code else ''}): {what}")
|
|
98
|
+
self.status = status
|
|
99
|
+
self.code = code
|
|
100
|
+
|
|
101
|
+
|
|
102
|
+
def _rejection_code(error: urllib.error.HTTPError) -> Optional[str]:
|
|
103
|
+
"""The router's error code from a refusal body ({"error_code": ..., "error": {"code": ...}}), if it sent one."""
|
|
104
|
+
try:
|
|
105
|
+
data = json.loads(error.read().decode("utf-8"))
|
|
106
|
+
except Exception:
|
|
107
|
+
return None
|
|
108
|
+
if not isinstance(data, dict):
|
|
109
|
+
return None
|
|
110
|
+
code = data.get("error_code")
|
|
111
|
+
if not isinstance(code, str):
|
|
112
|
+
nested = data.get("error")
|
|
113
|
+
code = nested.get("code") if isinstance(nested, dict) else nested
|
|
114
|
+
return code if isinstance(code, str) else None
|
|
115
|
+
|
|
116
|
+
|
|
84
117
|
def _unavailable(detail: str) -> PreflightDecision:
|
|
85
118
|
if _config.fail_mode == "closed":
|
|
86
119
|
raise GovernanceUnavailable(f"TokenMeter governance preflight unavailable ({detail}); failing closed")
|
|
@@ -99,7 +132,8 @@ def preflight(
|
|
|
99
132
|
estimated_input_tokens: Optional[int] = None,
|
|
100
133
|
max_output_tokens: Optional[int] = None,
|
|
101
134
|
) -> PreflightDecision:
|
|
102
|
-
"""Ask for a decision. Raises
|
|
135
|
+
"""Ask for a decision. Raises GovernanceUnavailable (fail_mode='closed') when the control plane cannot be
|
|
136
|
+
reached, and GovernanceRejected, whatever fail_mode is, when it refuses the key or the request."""
|
|
103
137
|
if not _config.router_url or not _config.ingest_key:
|
|
104
138
|
return _unavailable("router_url not configured -- pass router_url to tokenmeter.init()")
|
|
105
139
|
ctx = current()
|
|
@@ -125,6 +159,10 @@ def preflight(
|
|
|
125
159
|
return _unavailable(f"HTTP {resp.status}")
|
|
126
160
|
data = json.loads(resp.read().decode("utf-8"))
|
|
127
161
|
except urllib.error.HTTPError as e:
|
|
162
|
+
# 400/401/403 are answers, not an outage: the same request gets the same answer until the key or the
|
|
163
|
+
# request is fixed.
|
|
164
|
+
if e.code in (400, 401, 403):
|
|
165
|
+
raise GovernanceRejected(e.code, _rejection_code(e)) from None
|
|
128
166
|
return _unavailable(f"HTTP {e.code}")
|
|
129
167
|
except Exception as e: # network, timeout, bad JSON
|
|
130
168
|
return _unavailable(type(e).__name__)
|
|
@@ -1,91 +1,91 @@
|
|
|
1
|
-
"""SDK-side prompt topics (docs/PROMPT_INTELLIGENCE_AND_BUSINESS_UNITS_PLAN.md A-5). Same contract as the TypeScript
|
|
2
|
-
SDK's topic.ts.
|
|
3
|
-
|
|
4
|
-
Opt-in: pass ``optimization={"topic_classifier": fn}`` to ``tokenmeter.init`` (or ``action``) and each call is labelled
|
|
5
|
-
with ``tags["prompt_topic"]``, a topic KEY your classifier returns. The prompt text never leaves your process: the
|
|
6
|
-
classifier runs locally, and only a key matching ``^[a-z][a-z0-9_]{1,39}$`` is sent (anything else is dropped here,
|
|
7
|
-
and again at ingest).
|
|
8
|
-
|
|
9
|
-
The classifier must be synchronous and fast: it runs on the call path. A classifier that returns a coroutine, raises,
|
|
10
|
-
or returns something that is not a key is ignored and the call proceeds untouched. ``keyword_topic_classifier`` is a
|
|
11
|
-
small built-in one for the standard topic list (keywords only, no model).
|
|
12
|
-
"""
|
|
13
|
-
from __future__ import annotations
|
|
14
|
-
|
|
15
|
-
import inspect
|
|
16
|
-
import re
|
|
17
|
-
from typing import Any, Callable, Optional
|
|
18
|
-
|
|
19
|
-
TOPIC_KEY_RE = re.compile(r"^[a-z][a-z0-9_]{1,39}$")
|
|
20
|
-
TOPIC_TEXT_CHARS = 2_000
|
|
21
|
-
|
|
22
|
-
TopicClassifier = Callable[[str], Optional[str]]
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
def _parts_text(value: Any) -> Optional[str]:
|
|
26
|
-
if isinstance(value, str):
|
|
27
|
-
return value
|
|
28
|
-
if not isinstance(value, list):
|
|
29
|
-
return None
|
|
30
|
-
parts = [p["text"] for p in value if isinstance(p, dict) and p.get("type") in ("text", "input_text") and isinstance(p.get("text"), str)]
|
|
31
|
-
return "\n".join(parts) if parts else None
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
def latest_user_text(params: Any) -> Optional[str]:
|
|
35
|
-
"""The last user message of a chat / messages / responses request, or a plain prompt string."""
|
|
36
|
-
if not isinstance(params, dict):
|
|
37
|
-
return None
|
|
38
|
-
messages = params.get("messages", params.get("input"))
|
|
39
|
-
if isinstance(messages, str):
|
|
40
|
-
return messages
|
|
41
|
-
if isinstance(messages, list):
|
|
42
|
-
for message in reversed(messages):
|
|
43
|
-
if isinstance(message, dict) and message.get("role") == "user":
|
|
44
|
-
text = _parts_text(message.get("content"))
|
|
45
|
-
if text and text.strip():
|
|
46
|
-
return text
|
|
47
|
-
return None
|
|
48
|
-
prompt = params.get("prompt")
|
|
49
|
-
return prompt if isinstance(prompt, str) else None
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
def topic_tag(classifier: Any, params: Any) -> Optional[str]:
|
|
53
|
-
"""Runs the caller's classifier defensively; returns a valid key or None."""
|
|
54
|
-
if not callable(classifier):
|
|
55
|
-
return None
|
|
56
|
-
text = latest_user_text(params)
|
|
57
|
-
if not text or not text.strip():
|
|
58
|
-
return None
|
|
59
|
-
try:
|
|
60
|
-
key = classifier(text[:TOPIC_TEXT_CHARS])
|
|
61
|
-
except Exception:
|
|
62
|
-
return None
|
|
63
|
-
if inspect.isawaitable(key):
|
|
64
|
-
# An async classifier would add latency; it is ignored (and closed so it does not warn).
|
|
65
|
-
close = getattr(key, "close", None)
|
|
66
|
-
if callable(close):
|
|
67
|
-
close()
|
|
68
|
-
return None
|
|
69
|
-
return key if isinstance(key, str) and TOPIC_KEY_RE.match(key) else None
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
# Order matters: the first rule that matches wins (an error report about code is debugging, not code generation).
|
|
73
|
-
_KEYWORD_RULES = [
|
|
74
|
-
("debugging", re.compile(r"\b(error|exception|stack ?trace|traceback|bug|crash(es|ed)?|failing|fails|broken|typeerror|nullpointer|segfault|doesn'?t work)\b", re.I)),
|
|
75
|
-
("translation", re.compile(r"\b(translat(e|ion|ing)|in (french|spanish|german|italian|portuguese|japanese|chinese|korean|arabic|hindi|dutch))\b", re.I)),
|
|
76
|
-
("summarisation", re.compile(r"\b(summar(y|ise|ize|ising|izing)|tl;?dr|condense|key points|recap)\b", re.I)),
|
|
77
|
-
("data_extraction", re.compile(r"\b(extract|parse|pull out)\b[\s\S]{0,80}\b(fields?|json|table|entit(y|ies)|names?|dates?|values?|data)\b", re.I)),
|
|
78
|
-
("classification", re.compile(r"\b(classif(y|ication)|categori[sz]e|sentiment|label (this|these|each)|which category)\b", re.I)),
|
|
79
|
-
("customer_support", re.compile(r"\b(customer|ticket|refund|complaint|support request|reply to (this|the) (customer|user))\b", re.I)),
|
|
80
|
-
("content_writing", re.compile(r"\b(blog|article|newsletter|tagline|headline|marketing copy|social (media )?post|write an? (email|post|story))\b", re.I)),
|
|
81
|
-
("code_generation", re.compile(r"```|\b(write|implement|create|generate|build)\b[\s\S]{0,60}\b(function|class|script|code|query|sql|regex|component|endpoint|api|test)\b", re.I)),
|
|
82
|
-
("analysis_qa", re.compile(r"\b(explain|why|how (does|do|can)|what (is|are)|compare|analy[sz]e|pros and cons)\b|\?\s*$", re.I)),
|
|
83
|
-
]
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
def keyword_topic_classifier(text: str) -> str:
|
|
87
|
-
"""A keyword-only classifier for the standard topics. Coarse by design; supply your own for better labels."""
|
|
88
|
-
for key, pattern in _KEYWORD_RULES:
|
|
89
|
-
if pattern.search(text):
|
|
90
|
-
return key
|
|
91
|
-
return "other"
|
|
1
|
+
"""SDK-side prompt topics (docs/PROMPT_INTELLIGENCE_AND_BUSINESS_UNITS_PLAN.md A-5). Same contract as the TypeScript
|
|
2
|
+
SDK's topic.ts.
|
|
3
|
+
|
|
4
|
+
Opt-in: pass ``optimization={"topic_classifier": fn}`` to ``tokenmeter.init`` (or ``action``) and each call is labelled
|
|
5
|
+
with ``tags["prompt_topic"]``, a topic KEY your classifier returns. The prompt text never leaves your process: the
|
|
6
|
+
classifier runs locally, and only a key matching ``^[a-z][a-z0-9_]{1,39}$`` is sent (anything else is dropped here,
|
|
7
|
+
and again at ingest).
|
|
8
|
+
|
|
9
|
+
The classifier must be synchronous and fast: it runs on the call path. A classifier that returns a coroutine, raises,
|
|
10
|
+
or returns something that is not a key is ignored and the call proceeds untouched. ``keyword_topic_classifier`` is a
|
|
11
|
+
small built-in one for the standard topic list (keywords only, no model).
|
|
12
|
+
"""
|
|
13
|
+
from __future__ import annotations
|
|
14
|
+
|
|
15
|
+
import inspect
|
|
16
|
+
import re
|
|
17
|
+
from typing import Any, Callable, Optional
|
|
18
|
+
|
|
19
|
+
TOPIC_KEY_RE = re.compile(r"^[a-z][a-z0-9_]{1,39}$")
|
|
20
|
+
TOPIC_TEXT_CHARS = 2_000
|
|
21
|
+
|
|
22
|
+
TopicClassifier = Callable[[str], Optional[str]]
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
def _parts_text(value: Any) -> Optional[str]:
|
|
26
|
+
if isinstance(value, str):
|
|
27
|
+
return value
|
|
28
|
+
if not isinstance(value, list):
|
|
29
|
+
return None
|
|
30
|
+
parts = [p["text"] for p in value if isinstance(p, dict) and p.get("type") in ("text", "input_text") and isinstance(p.get("text"), str)]
|
|
31
|
+
return "\n".join(parts) if parts else None
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def latest_user_text(params: Any) -> Optional[str]:
|
|
35
|
+
"""The last user message of a chat / messages / responses request, or a plain prompt string."""
|
|
36
|
+
if not isinstance(params, dict):
|
|
37
|
+
return None
|
|
38
|
+
messages = params.get("messages", params.get("input"))
|
|
39
|
+
if isinstance(messages, str):
|
|
40
|
+
return messages
|
|
41
|
+
if isinstance(messages, list):
|
|
42
|
+
for message in reversed(messages):
|
|
43
|
+
if isinstance(message, dict) and message.get("role") == "user":
|
|
44
|
+
text = _parts_text(message.get("content"))
|
|
45
|
+
if text and text.strip():
|
|
46
|
+
return text
|
|
47
|
+
return None
|
|
48
|
+
prompt = params.get("prompt")
|
|
49
|
+
return prompt if isinstance(prompt, str) else None
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
def topic_tag(classifier: Any, params: Any) -> Optional[str]:
|
|
53
|
+
"""Runs the caller's classifier defensively; returns a valid key or None."""
|
|
54
|
+
if not callable(classifier):
|
|
55
|
+
return None
|
|
56
|
+
text = latest_user_text(params)
|
|
57
|
+
if not text or not text.strip():
|
|
58
|
+
return None
|
|
59
|
+
try:
|
|
60
|
+
key = classifier(text[:TOPIC_TEXT_CHARS])
|
|
61
|
+
except Exception:
|
|
62
|
+
return None
|
|
63
|
+
if inspect.isawaitable(key):
|
|
64
|
+
# An async classifier would add latency; it is ignored (and closed so it does not warn).
|
|
65
|
+
close = getattr(key, "close", None)
|
|
66
|
+
if callable(close):
|
|
67
|
+
close()
|
|
68
|
+
return None
|
|
69
|
+
return key if isinstance(key, str) and TOPIC_KEY_RE.match(key) else None
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
# Order matters: the first rule that matches wins (an error report about code is debugging, not code generation).
|
|
73
|
+
_KEYWORD_RULES = [
|
|
74
|
+
("debugging", re.compile(r"\b(error|exception|stack ?trace|traceback|bug|crash(es|ed)?|failing|fails|broken|typeerror|nullpointer|segfault|doesn'?t work)\b", re.I)),
|
|
75
|
+
("translation", re.compile(r"\b(translat(e|ion|ing)|in (french|spanish|german|italian|portuguese|japanese|chinese|korean|arabic|hindi|dutch))\b", re.I)),
|
|
76
|
+
("summarisation", re.compile(r"\b(summar(y|ise|ize|ising|izing)|tl;?dr|condense|key points|recap)\b", re.I)),
|
|
77
|
+
("data_extraction", re.compile(r"\b(extract|parse|pull out)\b[\s\S]{0,80}\b(fields?|json|table|entit(y|ies)|names?|dates?|values?|data)\b", re.I)),
|
|
78
|
+
("classification", re.compile(r"\b(classif(y|ication)|categori[sz]e|sentiment|label (this|these|each)|which category)\b", re.I)),
|
|
79
|
+
("customer_support", re.compile(r"\b(customer|ticket|refund|complaint|support request|reply to (this|the) (customer|user))\b", re.I)),
|
|
80
|
+
("content_writing", re.compile(r"\b(blog|article|newsletter|tagline|headline|marketing copy|social (media )?post|write an? (email|post|story))\b", re.I)),
|
|
81
|
+
("code_generation", re.compile(r"```|\b(write|implement|create|generate|build)\b[\s\S]{0,60}\b(function|class|script|code|query|sql|regex|component|endpoint|api|test)\b", re.I)),
|
|
82
|
+
("analysis_qa", re.compile(r"\b(explain|why|how (does|do|can)|what (is|are)|compare|analy[sz]e|pros and cons)\b|\?\s*$", re.I)),
|
|
83
|
+
]
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+
def keyword_topic_classifier(text: str) -> str:
|
|
87
|
+
"""A keyword-only classifier for the standard topics. Coarse by design; supply your own for better labels."""
|
|
88
|
+
for key, pattern in _KEYWORD_RULES:
|
|
89
|
+
if pattern.search(text):
|
|
90
|
+
return key
|
|
91
|
+
return "other"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: tokenmeter-sdk
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Summary: Token Meter Python SDK: wraps OpenAI/Anthropic/Gemini/Bedrock/Azure clients and meters token usage
|
|
5
5
|
Author: Saqlain Rashid
|
|
6
6
|
License-Expression: MIT
|
|
@@ -36,20 +36,6 @@ Dynamic: license-file
|
|
|
36
36
|
|
|
37
37
|
# token-meter (Python SDK)
|
|
38
38
|
|
|
39
|
-
## Read usage data
|
|
40
|
-
|
|
41
|
-
Create a separate read-only key under **Settings → API keys → Read API keys**. It cannot submit usage events.
|
|
42
|
-
|
|
43
|
-
```python
|
|
44
|
-
import os
|
|
45
|
-
from tokenmeter import TokenMeterReadClient
|
|
46
|
-
|
|
47
|
-
read_api = TokenMeterReadClient(os.environ["TOKEN_METER_READ_KEY"], os.environ["TOKEN_METER_DASHBOARD_URL"])
|
|
48
|
-
page = read_api.list_daily_usage("2026-10-01", "2026-10-31")
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
This returns daily **metered** usage/spend summaries, not provider invoice data. See [`docs/openapi.yaml`](../../docs/openapi.yaml) for pagination, fields, and limits.
|
|
52
|
-
|
|
53
39
|
In-process LLM usage capture for Python. `tokenmeter.wrap(client)` wraps a
|
|
54
40
|
provider client you already have, reads token usage out of the response
|
|
55
41
|
object it already returns (no extra network hop), normalizes it to a
|
|
@@ -128,6 +114,38 @@ client = tokenmeter.wrap_openai_compatible(
|
|
|
128
114
|
)
|
|
129
115
|
```
|
|
130
116
|
|
|
117
|
+
## Read usage data
|
|
118
|
+
|
|
119
|
+
Create a separate read-only key under **Settings → API keys → Read API keys**. It cannot submit usage events.
|
|
120
|
+
|
|
121
|
+
```python
|
|
122
|
+
import os
|
|
123
|
+
from tokenmeter import TokenMeterReadClient
|
|
124
|
+
|
|
125
|
+
read_api = TokenMeterReadClient(os.environ["TOKEN_METER_READ_KEY"], os.environ["TOKEN_METER_DASHBOARD_URL"])
|
|
126
|
+
page = read_api.list_daily_usage("2026-10-01", "2026-10-31")
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
This returns daily **metered** usage/spend summaries, not provider invoice data. Rows use camelCase keys (`pricedCostUsd`, `nextCursor`). See the [API reference](https://timewastevc.com/docs/reference#read-api) for pagination and fields.
|
|
130
|
+
|
|
131
|
+
## Governance preflight
|
|
132
|
+
|
|
133
|
+
For calls made to a provider directly (not through the Token Meter router), ask the router's control plane first
|
|
134
|
+
whether the call is allowed:
|
|
135
|
+
|
|
136
|
+
```python
|
|
137
|
+
tokenmeter.init(ingest_key="tl_live_...", router_url="https://router.timewastevc.com")
|
|
138
|
+
|
|
139
|
+
with tokenmeter.governed(provider="openai", model="gpt-4o", content=prompt, max_output_tokens=800):
|
|
140
|
+
client.chat.completions.create(model="gpt-4o", messages=messages, max_tokens=800)
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
Unlike metering, this is on the request path: it adds one HTTP call to the router (2 s timeout), raises
|
|
144
|
+
`tokenmeter.GovernanceDenied` on a denial (`governance_mode="advisory"` only records it), and raises
|
|
145
|
+
`tokenmeter.GovernanceUnavailable` when the router can't be reached (`governance_fail_mode="open"` proceeds
|
|
146
|
+
unprotected). A wrong or revoked ingest key (401, 403) or a malformed request (400) always raises
|
|
147
|
+
`tokenmeter.GovernanceRejected`, whatever the fail mode is: it is not an outage. See the [API reference](https://timewastevc.com/docs/reference#preflight).
|
|
148
|
+
|
|
131
149
|
## Provider support matrix
|
|
132
150
|
|
|
133
151
|
| Provider | Client | Paths wrapped | Streaming | Normalization confidence |
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{tokenmeter_sdk-0.2.0 → tokenmeter_sdk-0.2.1}/src/tokenmeter_sdk.egg-info/dependency_links.txt
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|