llm-agent-budget 0.1.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- llm_agent_budget-0.1.3/.gitignore +8 -0
- llm_agent_budget-0.1.3/CHANGELOG.md +54 -0
- llm_agent_budget-0.1.3/LICENSE +21 -0
- llm_agent_budget-0.1.3/PKG-INFO +149 -0
- llm_agent_budget-0.1.3/README.md +117 -0
- llm_agent_budget-0.1.3/docs/guide.md +146 -0
- llm_agent_budget-0.1.3/docs/integrating.md +83 -0
- llm_agent_budget-0.1.3/docs/pydantic-ai.md +91 -0
- llm_agent_budget-0.1.3/docs/strands.md +111 -0
- llm_agent_budget-0.1.3/examples/a2a_delegation.py +111 -0
- llm_agent_budget-0.1.3/examples/fanout.py +67 -0
- llm_agent_budget-0.1.3/examples/integrate.py +44 -0
- llm_agent_budget-0.1.3/examples/pydantic_ai_delegation.py +130 -0
- llm_agent_budget-0.1.3/pyproject.toml +50 -0
- llm_agent_budget-0.1.3/src/agent_budget/__init__.py +92 -0
- llm_agent_budget-0.1.3/src/agent_budget/_money.py +140 -0
- llm_agent_budget-0.1.3/src/agent_budget/a2a_plugin.py +349 -0
- llm_agent_budget-0.1.3/src/agent_budget/budget.py +512 -0
- llm_agent_budget-0.1.3/src/agent_budget/carriers.py +253 -0
- llm_agent_budget-0.1.3/src/agent_budget/context.py +425 -0
- llm_agent_budget-0.1.3/src/agent_budget/py.typed +0 -0
- llm_agent_budget-0.1.3/src/agent_budget/strands.py +246 -0
- llm_agent_budget-0.1.3/tests/fixtures/budget_context/invalid.json +235 -0
- llm_agent_budget-0.1.3/tests/fixtures/budget_context/normalize.json +235 -0
- llm_agent_budget-0.1.3/tests/fixtures/budget_context/roundtrip.json +181 -0
- llm_agent_budget-0.1.3/tests/test_a2a_plugin.py +360 -0
- llm_agent_budget-0.1.3/tests/test_budget.py +363 -0
- llm_agent_budget-0.1.3/tests/test_carriers.py +169 -0
- llm_agent_budget-0.1.3/tests/test_fixtures.py +75 -0
- llm_agent_budget-0.1.3/tests/test_hardening.py +245 -0
- llm_agent_budget-0.1.3/tests/test_strands.py +192 -0
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## [0.1.3] - 2026-11-17
|
|
4
|
+
|
|
5
|
+
The first release on PyPI, under the distribution name **llm-agent-budget**
|
|
6
|
+
(`pip install llm-agent-budget`). PyPI refused the name `agent-budget` as too
|
|
7
|
+
similar to an existing project. The import name is unchanged:
|
|
8
|
+
`import agent_budget`. 0.1.2 was tagged on GitHub but never reached PyPI.
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
- `agent_budget.strands` (extra `strands`, strands-agents 1.43 or later): `SpendCeiling`, a Strands plugin that caps what each invocation spends in USD. It reserves each model call's worst case (projected input plus the output-token cap) in `BeforeModelCallEvent`, cancels the call if it does not fit, and settles the reported usage in `AfterModelCallEvent`. An agent with a `SpendCeiling` that runs as a tool of another one gets its `usd` as a child slice of the caller's budget, so sub-agents started in one turn cannot be admitted for more than the caller has left. `budget=` draws every invocation from one shared `Budget`. `Price` prices a Strands `Usage` once per counter whether the provider folds cache tokens into `inputTokens` or reports them on top.
|
|
12
|
+
- `docs/strands.md` and `tests/test_strands.py` (real Strands agents with a scripted model, offline). Four researchers started in one turn under a $0.05 orchestrator: $0.078 without ceilings, $0.0468 with them.
|
|
13
|
+
|
|
14
|
+
## [0.1.2] - 2026-11-17
|
|
15
|
+
|
|
16
|
+
For projects deciding whether to depend on agent-budget. No behavior changes.
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
- `docs/integrating.md`: dependencies (none for the core), supported Python versions, typing, side effects, concurrency, how to make agent-budget an optional extra of your own package, the five-line reserve/settle integration, which entry point fits which stack (any client, sub-agents, pydantic-ai, A2A, baggage, MCP, a Rust service), and the stability policy.
|
|
20
|
+
- `examples/integrate.py`: that integration, runnable; CI runs it on every Python version.
|
|
21
|
+
- CI job `typecheck`: `mypy --strict` over the package (with the `a2a` extra) and the integration example.
|
|
22
|
+
|
|
23
|
+
### Fixed
|
|
24
|
+
- The package ships `py.typed` but did not pass `mypy --strict` (36 errors): unannotated parameters on the a2a plugins' hooks, `Budget` internals and `outbound` / `incurred_of` / `IncurredBooker`, and bare `dict` / `tuple` / `Token` types. Every public function is now fully annotated, so a strict type checker in a project that uses agent-budget sees real types instead of `Any`.
|
|
25
|
+
|
|
26
|
+
## [0.1.1] - 2026-11-17
|
|
27
|
+
|
|
28
|
+
The first version meant for PyPI; 0.1.0 was never uploaded.
|
|
29
|
+
|
|
30
|
+
### Fixed
|
|
31
|
+
- Untrusted amounts could raise `OverflowError` instead of `ContextError`: a JSON number such as `1e300`, or an integer too large for a float, escaped `BudgetContext.from_dict` and the a2a plugin's `read_context`, so a peer could make `before_execution` or `after_client_request` raise. Amounts are now range-checked before they are scaled, `from_dict` turns anything that slips past a field check into `ContextError`, and `read_context` logs and ignores every malformed payload.
|
|
32
|
+
- JSON integer amounts are read exactly (`18446744073` stays `"18446744073"`; through float it came out as `18446744072.999999488`), as llm-budget reads them.
|
|
33
|
+
- `IncurredBooker.book` raised `AmountError` on a large but valid report (the largest amount, or `cost + aggregatedChildCost` above it), after the remote work was done. It now books in nano-units without a float round trip and saturates, like llm-budget.
|
|
34
|
+
- Entering one `Budget` with `with` from two asyncio tasks or threads at once raised `ValueError` (the token stack lived on the shared object). Entries are now kept per context.
|
|
35
|
+
- `BudgetExecutionPlugin` gave a caller that granted $0 (an exhausted caller) the `default_usd` budget, and did the same for a budget in another currency. A $0 grant is now a $0 budget that refuses every reservation; a non-USD budget is logged and treated as $0.
|
|
36
|
+
- A `Scope` nobody closed kept its slice reserved in the parent forever. It is now closed when garbage-collected, with a `ResourceWarning`, matching llm-budget's drop of a `ScopeGuard`.
|
|
37
|
+
- A token count given as both `cacheRead` and `cache_read` (or `cacheWrite` and `cache_write`) is rejected, as llm-budget rejects it.
|
|
38
|
+
|
|
39
|
+
### Added
|
|
40
|
+
- `BudgetClientPlugin(per_call_usd=..., per_call_fraction=...)` opens a child scope for each call, sends that slice and closes it when the call ends, so parallel calls from one root budget are no longer each told the same remainder.
|
|
41
|
+
- `cap_reports=True` on `BudgetClientPlugin` and `BudgetClientInterceptor`, and `IncurredBooker(cap_nanos=...)` / `IncurredBooker.capped_at(scope)` with `excess_nanos`: book at most what the callee was given and flag the rest.
|
|
42
|
+
- `BudgetClientInterceptor` takes `deadline` and `token_limit`, like the plugin.
|
|
43
|
+
- `docs/pydantic-ai.md` and `examples/pydantic_ai_delegation.py`: a USD cap across parallel pydantic-ai delegates (a `WrapperModel` that reserves and settles every request, and a delegation tool that runs the sub-agent in a child scope), run offline in CI.
|
|
44
|
+
- `tests/test_hardening.py` and new cases in `tests/test_a2a_plugin.py` for each fix; the shared fixtures gain oversized JSON-number amounts, counts given under both names and an exact large whole-number amount.
|
|
45
|
+
|
|
46
|
+
## [0.1.0] - 2026-11-17
|
|
47
|
+
|
|
48
|
+
First release.
|
|
49
|
+
|
|
50
|
+
- `Budget` with `reserve` / `Reservation.settle` / `release` (exact nano-dollar accounting, thread-safe, asyncio-safe) and child scopes: `Budget.child(name, usd=... | fraction=...)` reserves a sub-agent's slice from its parent before it starts; everything the child books is booked on every ancestor at once; closing it refunds the unused part. `ScopeClosed`, `BudgetKilled` (kill switches stop a whole subtree), `DepthExceeded`, `BudgetExhausted`. `ScopeReceipt` with own and child cost. `current_budget()` follows `with` blocks per thread and asyncio task.
|
|
51
|
+
- `BudgetContext` in the shape of the A2A budget extension draft (a2aproject/A2A#2121): `budget{cost, tokens, deadline}`, `incurred{cost, aggregatedChildCost, tokens, delegations}`, plus `reservation{id, amount, currency, expires}`. Money as decimal strings, absent never written as zero. `outbound(scope)`, `incurred_of(budget)`, `IncurredBooker` (books cumulative reports once).
|
|
52
|
+
- Carriers: A2A metadata (`to_a2a_metadata`, `from_a2a_metadata`), W3C baggage (`to_baggage`, `from_baggage`, `merge_baggage`) and MCP `_meta.baggage` (`inject_mcp_meta`, `extract_mcp_meta`).
|
|
53
|
+
- `agent_budget.a2a_plugin` (extra `a2a`): `BudgetClientPlugin`, `BudgetExecutionPlugin` and `BudgetClientInterceptor` for a2a-sdk 1.3+.
|
|
54
|
+
- Golden fixtures shared byte for byte with the Rust crate llm-budget 0.8.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Matthew Busel
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: llm-agent-budget
|
|
3
|
+
Version: 0.1.3
|
|
4
|
+
Summary: Hard USD budgets for agents that delegate: reserve/settle child scopes for parallel sub-agents, and the A2A budget extension over A2A metadata, W3C baggage and MCP _meta
|
|
5
|
+
Project-URL: Homepage, https://github.com/Mattbusel/agent-budget
|
|
6
|
+
Project-URL: Source, https://github.com/Mattbusel/agent-budget
|
|
7
|
+
Project-URL: Issues, https://github.com/Mattbusel/agent-budget/issues
|
|
8
|
+
Project-URL: Changelog, https://github.com/Mattbusel/agent-budget/blob/main/CHANGELOG.md
|
|
9
|
+
Project-URL: Rust crate, https://crates.io/crates/llm-budget
|
|
10
|
+
Author-email: Matthew Charles Busel <mattbusel@gmail.com>
|
|
11
|
+
License-Expression: MIT
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Keywords: a2a,agents,budget,cost,llm,mcp,multi-agent,opentelemetry,strands-agents
|
|
14
|
+
Classifier: Development Status :: 4 - Beta
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: Operating System :: OS Independent
|
|
17
|
+
Classifier: Programming Language :: Python :: 3
|
|
18
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
19
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
20
|
+
Classifier: Topic :: Software Development :: Libraries
|
|
21
|
+
Classifier: Typing :: Typed
|
|
22
|
+
Requires-Python: >=3.9
|
|
23
|
+
Provides-Extra: a2a
|
|
24
|
+
Requires-Dist: a2a-sdk<2,>=1.3; extra == 'a2a'
|
|
25
|
+
Provides-Extra: strands
|
|
26
|
+
Requires-Dist: strands-agents>=1.43; extra == 'strands'
|
|
27
|
+
Provides-Extra: test
|
|
28
|
+
Requires-Dist: hypothesis>=6.100; extra == 'test'
|
|
29
|
+
Requires-Dist: pytest-asyncio>=0.23; extra == 'test'
|
|
30
|
+
Requires-Dist: pytest>=8; extra == 'test'
|
|
31
|
+
Description-Content-Type: text/markdown
|
|
32
|
+
|
|
33
|
+
# agent-budget
|
|
34
|
+
|
|
35
|
+
Hard USD budgets for agents that delegate.
|
|
36
|
+
|
|
37
|
+
When an orchestrator hands work to several sub-agents at once, each one
|
|
38
|
+
checks its spending against the same "remaining" figure, none of them sees
|
|
39
|
+
the others, and together they spend a multiple of what was left. agent-budget
|
|
40
|
+
fixes that at the delegation boundary: each sub-agent's slice is **reserved
|
|
41
|
+
from the parent before it starts**, every amount it books is booked on the
|
|
42
|
+
parent at the same moment, and the unused part goes back when it finishes.
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
pip install llm-agent-budget # no dependencies
|
|
46
|
+
pip install "llm-agent-budget[a2a]" # plus the a2a-python plugins
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## Child scopes
|
|
50
|
+
|
|
51
|
+
```python
|
|
52
|
+
from agent_budget import Budget
|
|
53
|
+
|
|
54
|
+
orchestrator = Budget(1.00, name="orchestrator")
|
|
55
|
+
|
|
56
|
+
with orchestrator.child("researcher", usd=0.60) as research:
|
|
57
|
+
hold = research.reserve(0.06) # before the model call; refused if it does not fit
|
|
58
|
+
# ... call the model ...
|
|
59
|
+
hold.settle(0.052) # what it really cost
|
|
60
|
+
|
|
61
|
+
with research.child("fetcher", fraction=0.5) as fetch: # half of what research has left
|
|
62
|
+
fetch.reserve(0.01).settle(0.01)
|
|
63
|
+
|
|
64
|
+
print(orchestrator.spent_usd) # 0.062: every level, as it happens
|
|
65
|
+
print(orchestrator.remaining_usd) # 0.938: the unused slice came back on exit
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
- A slice that does not fit raises `BudgetExhausted` before the sub-agent
|
|
69
|
+
spends anything.
|
|
70
|
+
- A closed scope (and everything below it) raises `ScopeClosed`; calls
|
|
71
|
+
already in flight still settle.
|
|
72
|
+
- `kill()` stops a scope and its whole subtree. `max_depth` (default 8)
|
|
73
|
+
raises `DepthExceeded`.
|
|
74
|
+
- `own_cost_usd` + `child_cost_usd` == `spent_usd` on every scope.
|
|
75
|
+
- Thread-safe (one lock per tree) and safe in asyncio tasks;
|
|
76
|
+
`current_budget()` is the innermost scope entered with `with` in the
|
|
77
|
+
current thread or task. One budget can be entered by many threads and
|
|
78
|
+
tasks at once.
|
|
79
|
+
|
|
80
|
+
## Across agents: A2A, W3C baggage, MCP
|
|
81
|
+
|
|
82
|
+
`BudgetContext` is the budget a caller hands down and the cost a callee
|
|
83
|
+
reports back, in the shape of the
|
|
84
|
+
[A2A budget extension draft](https://github.com/a2aproject/A2A/issues/2121)
|
|
85
|
+
(`budget{cost, tokens, deadline}`, `incurred{cost, aggregatedChildCost,
|
|
86
|
+
tokens, delegations}`), plus a `reservation{id, amount, currency, expires}`
|
|
87
|
+
for the slice the caller reserved.
|
|
88
|
+
|
|
89
|
+
```python
|
|
90
|
+
from agent_budget import Budget, outbound, to_a2a_metadata, from_a2a_metadata, to_baggage
|
|
91
|
+
|
|
92
|
+
with Budget(1.00).child("remote", usd=0.30) as slice_:
|
|
93
|
+
ctx = outbound(slice_)
|
|
94
|
+
metadata = to_a2a_metadata(ctx) # {"https://a2a-protocol.org/extensions/budget/v1": {...}}
|
|
95
|
+
header = to_baggage(ctx) # "llm.budget.cost.amount=0.3,llm.budget.cost.currency=USD,..."
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
- **A2A**: the whole context under the extension URI in request, message or
|
|
99
|
+
task metadata.
|
|
100
|
+
- **W3C baggage**: `llm.budget.*` members, so anything that already
|
|
101
|
+
propagates OpenTelemetry baggage carries the budget.
|
|
102
|
+
- **MCP**: MCP reserves `params._meta.baggage` for W3C baggage (SEP-414);
|
|
103
|
+
`inject_mcp_meta` / `extract_mcp_meta` put the same members there.
|
|
104
|
+
|
|
105
|
+
Money is a decimal string on the wire, exact to the nano-dollar. Fields
|
|
106
|
+
nobody measured are left out, never written as zero.
|
|
107
|
+
|
|
108
|
+
## a2a-python plugins
|
|
109
|
+
|
|
110
|
+
```python
|
|
111
|
+
from a2a import plugins
|
|
112
|
+
from agent_budget.a2a_plugin import BudgetClientPlugin, BudgetExecutionPlugin
|
|
113
|
+
|
|
114
|
+
# Caller: requests made inside a scope carry its budget; replies are booked on it.
|
|
115
|
+
plugins.set_global(plugins.sequential(BudgetClientPlugin()))
|
|
116
|
+
with orchestrator.child("remote-researcher", usd=0.30):
|
|
117
|
+
async for event in client.send_message(request):
|
|
118
|
+
...
|
|
119
|
+
|
|
120
|
+
# Callee: each execution runs inside a Budget of what the caller allowed,
|
|
121
|
+
# and every task update reports the cumulative cost back.
|
|
122
|
+
handler = DefaultRequestHandlerV2(..., plugins=plugins.sequential(BudgetExecutionPlugin(max_usd=5.00)))
|
|
123
|
+
# inside AgentExecutor.execute: current_budget().reserve(...).settle(...)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
## Strands Agents
|
|
127
|
+
|
|
128
|
+
```python
|
|
129
|
+
from strands import Agent
|
|
130
|
+
from agent_budget.strands import Price, SpendCeiling
|
|
131
|
+
|
|
132
|
+
agent = Agent(tools=[researcher], plugins=[SpendCeiling(usd=0.50, price=Price(input=3.00, output=15.00))])
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
A Strands plugin (`pip install "llm-agent-budget[strands]"`): each invocation can
|
|
136
|
+
spend at most `usd`, every model call reserves its worst case before it is
|
|
137
|
+
sent, and an agent used as a tool gets its slice from the caller's budget.
|
|
138
|
+
See [docs/strands.md](docs/strands.md).
|
|
139
|
+
|
|
140
|
+
## Same format as the Rust crate
|
|
141
|
+
|
|
142
|
+
[llm-budget](https://crates.io/crates/llm-budget) 0.8 (Rust) has the same
|
|
143
|
+
scopes and the same contexts. Both test suites read one set of golden
|
|
144
|
+
fixtures (`tests/fixtures/budget_context/`), so a Rust orchestrator and a
|
|
145
|
+
Python agent agree to the last digit.
|
|
146
|
+
|
|
147
|
+
More: [docs/guide.md](docs/guide.md). Adding it to another project (dependencies, typing, stability, the five-line integration): [docs/integrating.md](docs/integrating.md).
|
|
148
|
+
|
|
149
|
+
MIT license.
|
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# agent-budget
|
|
2
|
+
|
|
3
|
+
Hard USD budgets for agents that delegate.
|
|
4
|
+
|
|
5
|
+
When an orchestrator hands work to several sub-agents at once, each one
|
|
6
|
+
checks its spending against the same "remaining" figure, none of them sees
|
|
7
|
+
the others, and together they spend a multiple of what was left. agent-budget
|
|
8
|
+
fixes that at the delegation boundary: each sub-agent's slice is **reserved
|
|
9
|
+
from the parent before it starts**, every amount it books is booked on the
|
|
10
|
+
parent at the same moment, and the unused part goes back when it finishes.
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
pip install llm-agent-budget # no dependencies
|
|
14
|
+
pip install "llm-agent-budget[a2a]" # plus the a2a-python plugins
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Child scopes
|
|
18
|
+
|
|
19
|
+
```python
|
|
20
|
+
from agent_budget import Budget
|
|
21
|
+
|
|
22
|
+
orchestrator = Budget(1.00, name="orchestrator")
|
|
23
|
+
|
|
24
|
+
with orchestrator.child("researcher", usd=0.60) as research:
|
|
25
|
+
hold = research.reserve(0.06) # before the model call; refused if it does not fit
|
|
26
|
+
# ... call the model ...
|
|
27
|
+
hold.settle(0.052) # what it really cost
|
|
28
|
+
|
|
29
|
+
with research.child("fetcher", fraction=0.5) as fetch: # half of what research has left
|
|
30
|
+
fetch.reserve(0.01).settle(0.01)
|
|
31
|
+
|
|
32
|
+
print(orchestrator.spent_usd) # 0.062: every level, as it happens
|
|
33
|
+
print(orchestrator.remaining_usd) # 0.938: the unused slice came back on exit
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
- A slice that does not fit raises `BudgetExhausted` before the sub-agent
|
|
37
|
+
spends anything.
|
|
38
|
+
- A closed scope (and everything below it) raises `ScopeClosed`; calls
|
|
39
|
+
already in flight still settle.
|
|
40
|
+
- `kill()` stops a scope and its whole subtree. `max_depth` (default 8)
|
|
41
|
+
raises `DepthExceeded`.
|
|
42
|
+
- `own_cost_usd` + `child_cost_usd` == `spent_usd` on every scope.
|
|
43
|
+
- Thread-safe (one lock per tree) and safe in asyncio tasks;
|
|
44
|
+
`current_budget()` is the innermost scope entered with `with` in the
|
|
45
|
+
current thread or task. One budget can be entered by many threads and
|
|
46
|
+
tasks at once.
|
|
47
|
+
|
|
48
|
+
## Across agents: A2A, W3C baggage, MCP
|
|
49
|
+
|
|
50
|
+
`BudgetContext` is the budget a caller hands down and the cost a callee
|
|
51
|
+
reports back, in the shape of the
|
|
52
|
+
[A2A budget extension draft](https://github.com/a2aproject/A2A/issues/2121)
|
|
53
|
+
(`budget{cost, tokens, deadline}`, `incurred{cost, aggregatedChildCost,
|
|
54
|
+
tokens, delegations}`), plus a `reservation{id, amount, currency, expires}`
|
|
55
|
+
for the slice the caller reserved.
|
|
56
|
+
|
|
57
|
+
```python
|
|
58
|
+
from agent_budget import Budget, outbound, to_a2a_metadata, from_a2a_metadata, to_baggage
|
|
59
|
+
|
|
60
|
+
with Budget(1.00).child("remote", usd=0.30) as slice_:
|
|
61
|
+
ctx = outbound(slice_)
|
|
62
|
+
metadata = to_a2a_metadata(ctx) # {"https://a2a-protocol.org/extensions/budget/v1": {...}}
|
|
63
|
+
header = to_baggage(ctx) # "llm.budget.cost.amount=0.3,llm.budget.cost.currency=USD,..."
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
- **A2A**: the whole context under the extension URI in request, message or
|
|
67
|
+
task metadata.
|
|
68
|
+
- **W3C baggage**: `llm.budget.*` members, so anything that already
|
|
69
|
+
propagates OpenTelemetry baggage carries the budget.
|
|
70
|
+
- **MCP**: MCP reserves `params._meta.baggage` for W3C baggage (SEP-414);
|
|
71
|
+
`inject_mcp_meta` / `extract_mcp_meta` put the same members there.
|
|
72
|
+
|
|
73
|
+
Money is a decimal string on the wire, exact to the nano-dollar. Fields
|
|
74
|
+
nobody measured are left out, never written as zero.
|
|
75
|
+
|
|
76
|
+
## a2a-python plugins
|
|
77
|
+
|
|
78
|
+
```python
|
|
79
|
+
from a2a import plugins
|
|
80
|
+
from agent_budget.a2a_plugin import BudgetClientPlugin, BudgetExecutionPlugin
|
|
81
|
+
|
|
82
|
+
# Caller: requests made inside a scope carry its budget; replies are booked on it.
|
|
83
|
+
plugins.set_global(plugins.sequential(BudgetClientPlugin()))
|
|
84
|
+
with orchestrator.child("remote-researcher", usd=0.30):
|
|
85
|
+
async for event in client.send_message(request):
|
|
86
|
+
...
|
|
87
|
+
|
|
88
|
+
# Callee: each execution runs inside a Budget of what the caller allowed,
|
|
89
|
+
# and every task update reports the cumulative cost back.
|
|
90
|
+
handler = DefaultRequestHandlerV2(..., plugins=plugins.sequential(BudgetExecutionPlugin(max_usd=5.00)))
|
|
91
|
+
# inside AgentExecutor.execute: current_budget().reserve(...).settle(...)
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## Strands Agents
|
|
95
|
+
|
|
96
|
+
```python
|
|
97
|
+
from strands import Agent
|
|
98
|
+
from agent_budget.strands import Price, SpendCeiling
|
|
99
|
+
|
|
100
|
+
agent = Agent(tools=[researcher], plugins=[SpendCeiling(usd=0.50, price=Price(input=3.00, output=15.00))])
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
A Strands plugin (`pip install "llm-agent-budget[strands]"`): each invocation can
|
|
104
|
+
spend at most `usd`, every model call reserves its worst case before it is
|
|
105
|
+
sent, and an agent used as a tool gets its slice from the caller's budget.
|
|
106
|
+
See [docs/strands.md](docs/strands.md).
|
|
107
|
+
|
|
108
|
+
## Same format as the Rust crate
|
|
109
|
+
|
|
110
|
+
[llm-budget](https://crates.io/crates/llm-budget) 0.8 (Rust) has the same
|
|
111
|
+
scopes and the same contexts. Both test suites read one set of golden
|
|
112
|
+
fixtures (`tests/fixtures/budget_context/`), so a Rust orchestrator and a
|
|
113
|
+
Python agent agree to the last digit.
|
|
114
|
+
|
|
115
|
+
More: [docs/guide.md](docs/guide.md). Adding it to another project (dependencies, typing, stability, the five-line integration): [docs/integrating.md](docs/integrating.md).
|
|
116
|
+
|
|
117
|
+
MIT license.
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# agent-budget guide
|
|
2
|
+
|
|
3
|
+
## Why child scopes
|
|
4
|
+
|
|
5
|
+
An orchestrator that starts several sub-agents at once and tells each one
|
|
6
|
+
"you have $X left" gives every sub-agent a figure that is valid when it reads
|
|
7
|
+
it and stale the moment a sibling spends. Each one stays under $X; together
|
|
8
|
+
they spend a multiple of it. Measured with `examples/fanout.py` (four
|
|
9
|
+
sub-agents, eight gpt-4o-sized calls each, $1.00 cap):
|
|
10
|
+
|
|
11
|
+
| Setup | Calls | Spent |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| remainder passed down | 32 | $1.664 (66% over the cap) |
|
|
14
|
+
| `parent.child(.., usd=0.25)` per sub-agent | 16 | $0.832 |
|
|
15
|
+
|
|
16
|
+
`tests/test_budget.py` has the two-thread version of the same overshoot
|
|
17
|
+
($1.60 on a $1.00 cap) and the same two delegates under child scopes (one
|
|
18
|
+
gets its slice, the other is refused before it spends anything).
|
|
19
|
+
|
|
20
|
+
## The rules
|
|
21
|
+
|
|
22
|
+
1. `Budget.reserve(estimate)` holds headroom before a call, atomically. If it
|
|
23
|
+
does not fit, `BudgetExhausted` is raised and the call is never made.
|
|
24
|
+
2. `Reservation.settle(actual)` replaces the hold with the real cost, even
|
|
25
|
+
past the cap (that money is already spent). `release()` drops it. Use
|
|
26
|
+
`with budget.reserve(x) as hold:` around the call so a cancelled task or
|
|
27
|
+
an exception releases the hold at once; a reservation garbage-collected
|
|
28
|
+
unsettled is also released, but only when the collector gets to it.
|
|
29
|
+
3. `Budget.child(name, usd=... | fraction=...)` reserves the child's whole
|
|
30
|
+
slice from the parent first. A `fraction` is of what the parent has left
|
|
31
|
+
at that moment.
|
|
32
|
+
4. Everything a child books is booked on every ancestor at the same moment,
|
|
33
|
+
out of the child's hold.
|
|
34
|
+
5. Closing a child (`with` exit, `close()`) stops new work in it and its
|
|
35
|
+
subtree (`ScopeClosed`) and refunds the part of its hold no in-flight call
|
|
36
|
+
needs. A scope nobody closed is closed when it is garbage-collected (like
|
|
37
|
+
dropping llm-budget's `ScopeGuard`), with a `ResourceWarning`; an open
|
|
38
|
+
reservation keeps its scope alive, so its hold is never cut short.
|
|
39
|
+
6. `kill(reason)` stops a scope and its subtree (`BudgetKilled`).
|
|
40
|
+
7. Depth is capped (`max_depth`, default 8; `DepthExceeded`).
|
|
41
|
+
|
|
42
|
+
For every scope, under any interleaving: open holds plus everything booked
|
|
43
|
+
never exceed its limit through admission. The only way past is a settlement
|
|
44
|
+
above its reservation, which is booked in full and reported (`over_cap`,
|
|
45
|
+
`ScopeReceipt.over_hold`). `tests/test_budget.py` checks this with 64
|
|
46
|
+
threads and a monitor, an asyncio fan-out with a cancelled task, and a
|
|
47
|
+
Hypothesis property test over random scope trees.
|
|
48
|
+
|
|
49
|
+
## Contexts on the wire
|
|
50
|
+
|
|
51
|
+
```json
|
|
52
|
+
{
|
|
53
|
+
"budget": {"cost": {"amount": "0.3", "currency": "USD"}, "tokens": {"limit": 5000}, "deadline": "2026-12-01T00:00:00Z"},
|
|
54
|
+
"reservation": {"id": "rsv_a2dcda7352c5fe3c882a05e2b66a162a", "amount": "0.3", "currency": "USD"},
|
|
55
|
+
"incurred": {
|
|
56
|
+
"cost": {"amount": "0.04", "currency": "USD", "source": "local"},
|
|
57
|
+
"aggregatedChildCost": {"amount": "0.02", "currency": "USD"},
|
|
58
|
+
"tokens": {"input": 1200, "output": 340, "cacheRead": 1000},
|
|
59
|
+
"delegations": 1
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
- `budget` and `incurred` follow the A2A budget extension draft,
|
|
65
|
+
[a2aproject/A2A#2121](https://github.com/a2aproject/A2A/issues/2121),
|
|
66
|
+
under its proposed URI `https://a2a-protocol.org/extensions/budget/v1`.
|
|
67
|
+
The draft is still open. Its author has said money will be decimal strings
|
|
68
|
+
(done), that `aggregatedChildCost` is a provisional name, and that token
|
|
69
|
+
counts gain cache fields. agent-budget follows the draft in minor releases
|
|
70
|
+
and keeps reading what it wrote before.
|
|
71
|
+
- `reservation` is this package's addition: the slice the caller reserved,
|
|
72
|
+
with an id unique to that reservation (never shared between siblings, so
|
|
73
|
+
a callee cannot learn how many siblings it has).
|
|
74
|
+
- `incurred.cost` is the reporter's own spend. `aggregatedChildCost` is its
|
|
75
|
+
delegates'. This split matches the OpenTelemetry GenAI cost proposal
|
|
76
|
+
(open-telemetry/semantic-conventions-genai#443), where an operation's cost
|
|
77
|
+
must not include nested operations.
|
|
78
|
+
- Absent is not zero: what was not measured is left out.
|
|
79
|
+
- Reports are cumulative; `IncurredBooker` books only their growth, so a
|
|
80
|
+
resent final task is not booked twice. `IncurredBooker.capped_at(scope)`
|
|
81
|
+
books at most what the scope holds and keeps the rest in `excess_nanos`.
|
|
82
|
+
Amounts saturate at the largest the wire carries; a report never makes
|
|
83
|
+
`book` raise.
|
|
84
|
+
|
|
85
|
+
### W3C baggage and MCP
|
|
86
|
+
|
|
87
|
+
The outbound half also travels as `llm.budget.*` baggage members:
|
|
88
|
+
|
|
89
|
+
```
|
|
90
|
+
llm.budget.cost.amount=0.3,llm.budget.cost.currency=USD,llm.budget.reservation.id=rsv_...,llm.budget.reservation.amount=0.3,llm.budget.reservation.currency=USD
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
MCP reserves `params._meta.baggage` for W3C baggage (SEP-414), so
|
|
94
|
+
`inject_mcp_meta(meta, ctx)` on the client and `extract_mcp_meta(meta)` on
|
|
95
|
+
the server carry a budget into any MCP tool call without an MCP change.
|
|
96
|
+
Readers reject malformed input with `ContextError` (and never anything else:
|
|
97
|
+
`tests/test_carriers.py` and `tests/test_hardening.py` fuzz them with
|
|
98
|
+
Hypothesis, including floats up to `1e308` and integers of hundreds of
|
|
99
|
+
digits). JSON integer amounts are read exactly, as llm-budget reads them.
|
|
100
|
+
|
|
101
|
+
## a2a-python
|
|
102
|
+
|
|
103
|
+
`agent_budget.a2a_plugin` (install the `a2a` extra; a2a-sdk 1.3+):
|
|
104
|
+
|
|
105
|
+
- `BudgetClientPlugin(budget=None, deadline=None, token_limit=None,
|
|
106
|
+
per_call_usd=None, per_call_fraction=None, cap_reports=False)`: for
|
|
107
|
+
`send_message` / `send_message_streaming` calls made while a scope is
|
|
108
|
+
current (or with a fixed `budget=`), writes the outbound context into the
|
|
109
|
+
request metadata, asks for the extension in `A2A-Extensions`, and books
|
|
110
|
+
each reply's `incurred` on the scope. Without a per-call slice, parallel
|
|
111
|
+
calls from one budget are each told the same remainder: open a child
|
|
112
|
+
scope per delegation, or set `per_call_usd` / `per_call_fraction` and the
|
|
113
|
+
plugin opens a child scope for each call and closes it when the call
|
|
114
|
+
ends (a slice that does not fit raises `BudgetExhausted` and the call is
|
|
115
|
+
not sent). `cap_reports=True` books at most what the call's scope holds.
|
|
116
|
+
- `BudgetExecutionPlugin(default_usd=None, max_usd=None, source="local")`:
|
|
117
|
+
reads the budget from the request (request metadata first, then message
|
|
118
|
+
metadata), opens a `Budget` of that amount (capped by `max_usd`; requests
|
|
119
|
+
without one get `default_usd` or no budget; a grant of $0 is a $0 budget
|
|
120
|
+
that refuses every reservation, and a budget in a currency other than USD
|
|
121
|
+
is logged and treated as $0), makes it `current_budget()`
|
|
122
|
+
inside `AgentExecutor.execute`, and stamps the cumulative `incurred`
|
|
123
|
+
report on every task, status update and message the agent emits.
|
|
124
|
+
- `BudgetClientInterceptor(budget=None, deadline=None, token_limit=None,
|
|
125
|
+
cap_reports=False)`: the client half for code that passes interceptors to
|
|
126
|
+
`ClientFactory.create`. Per-call slices need the plugin API.
|
|
127
|
+
|
|
128
|
+
`tests/test_a2a_plugin.py` runs all of this over real JSON-RPC and HTTP+JSON
|
|
129
|
+
transports (Starlette in-process): the budget arrives, the extension header
|
|
130
|
+
arrives, the callee spends inside it, reports are cumulative, the caller
|
|
131
|
+
books once, three concurrent delegations from one orchestrator add up.
|
|
132
|
+
`examples/a2a_delegation.py` is the same flow as a script.
|
|
133
|
+
|
|
134
|
+
## pydantic-ai
|
|
135
|
+
|
|
136
|
+
[pydantic-ai.md](pydantic-ai.md): a model wrapper that reserves and settles
|
|
137
|
+
every request on `current_budget()`, and a delegation tool that runs each
|
|
138
|
+
sub-agent in a child scope, so parallel delegates share one USD cap.
|
|
139
|
+
|
|
140
|
+
## Interop with llm-budget (Rust)
|
|
141
|
+
|
|
142
|
+
[llm-budget](https://crates.io/crates/llm-budget) 0.8 has the same scopes
|
|
143
|
+
(`AgentBudget::child`, `Limit::Usd`, `Limit::Fraction`) and the same
|
|
144
|
+
contexts (`llm_budget::context`). Both libraries read the same golden
|
|
145
|
+
fixtures in `tests/fixtures/budget_context/`; CI checks that the two copies
|
|
146
|
+
are identical.
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# Integrating agent-budget into another project
|
|
2
|
+
|
|
3
|
+
For maintainers deciding whether to depend on agent-budget, and for anyone
|
|
4
|
+
adding a hard USD cap to an agent framework, an orchestrator or an A2A / MCP
|
|
5
|
+
service. The snippets here run in CI (`examples/integrate.py`, also checked
|
|
6
|
+
with `mypy --strict`).
|
|
7
|
+
|
|
8
|
+
## At a glance
|
|
9
|
+
|
|
10
|
+
| | |
|
|
11
|
+
|---|---|
|
|
12
|
+
| License | MIT ([LICENSE](../LICENSE)) |
|
|
13
|
+
| Python | 3.9+ for the core, 3.10+ for the `a2a` extra (a2a-sdk's own minimum). Tested on 3.9 to 3.13 in CI. |
|
|
14
|
+
| Dependencies | None. The core is pure Python, standard library only. `a2a` extra: a2a-sdk 1.3+. |
|
|
15
|
+
| Typing | Ships `py.typed`; the package passes `mypy --strict` in CI. |
|
|
16
|
+
| Side effects | No network, no files, no threads, no global state beyond a `contextvars` variable for `current_budget()`. Nothing happens at import. |
|
|
17
|
+
| Concurrency | One lock per budget tree; safe from threads and asyncio tasks. One budget can be entered by many tasks at once. |
|
|
18
|
+
| Money | Exact integer nano-dollars inside, never float accumulation. |
|
|
19
|
+
| Maintainer | Matthew Busel ([Tensorust LLC](https://tensorust.com/open-source/)). Issues and PRs on [GitHub](https://github.com/Mattbusel/agent-budget). |
|
|
20
|
+
|
|
21
|
+
## Add it
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
pip install llm-agent-budget # no dependencies
|
|
25
|
+
pip install "llm-agent-budget[a2a]" # plus the a2a-sdk plugins
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
As an optional extra of your own package:
|
|
29
|
+
|
|
30
|
+
```toml
|
|
31
|
+
[project.optional-dependencies]
|
|
32
|
+
budget = ["llm-agent-budget>=0.1.3,<0.2"]
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
and import it lazily where the cap is applied, so users who do not ask for
|
|
36
|
+
budgets never import it.
|
|
37
|
+
|
|
38
|
+
## The five-line integration
|
|
39
|
+
|
|
40
|
+
Works around any client. Reserve the worst case before the call, settle the
|
|
41
|
+
real cost after it:
|
|
42
|
+
|
|
43
|
+
```python
|
|
44
|
+
from agent_budget import Budget
|
|
45
|
+
|
|
46
|
+
budget = Budget(5.00, name="my-agent") # $5 hard cap
|
|
47
|
+
with budget.reserve(0.01) as hold: # raises BudgetExhausted if it would not fit
|
|
48
|
+
cost = call_model("...") # the real call, with any client
|
|
49
|
+
hold.settle(cost) # books what it really cost
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
- `BudgetExhausted`, `BudgetKilled` and `ScopeClosed` all derive from
|
|
53
|
+
`BudgetError`; catch that one to handle every refusal. The call is never
|
|
54
|
+
made.
|
|
55
|
+
- Leaving the `with` block without settling (an exception, a cancelled task)
|
|
56
|
+
releases the hold, so nothing is booked for a call that did not happen.
|
|
57
|
+
- `budget.charge_force(cost)` books a cost you only hear about afterwards,
|
|
58
|
+
even past the cap.
|
|
59
|
+
|
|
60
|
+
## Pick the integration point
|
|
61
|
+
|
|
62
|
+
| Your code has | Use | See |
|
|
63
|
+
|---|---|---|
|
|
64
|
+
| Any client or SDK | `Budget.reserve` + `Reservation.settle` | above, `examples/integrate.py` |
|
|
65
|
+
| An orchestrator with sub-agents | `budget.child(name, usd=...)` or `fraction=...` | `examples/fanout.py`, [guide.md](guide.md) |
|
|
66
|
+
| Code that should find the active budget itself | `current_budget()` inside `with budget:` (per thread and asyncio task) | [guide.md](guide.md) |
|
|
67
|
+
| pydantic-ai | a `WrapperModel` that reserves and settles each request, delegates in child scopes | [pydantic-ai.md](pydantic-ai.md) |
|
|
68
|
+
| A2A (a2a-sdk 1.3+) | `BudgetClientPlugin` (caller) and `BudgetExecutionPlugin` (callee), or `BudgetClientInterceptor` | `examples/a2a_delegation.py` |
|
|
69
|
+
| OpenTelemetry baggage | `to_baggage` / `from_baggage` / `merge_baggage` (`llm.budget.*` members) | README |
|
|
70
|
+
| MCP tool servers | `inject_mcp_meta` / `extract_mcp_meta` on `params._meta` | README |
|
|
71
|
+
| A Rust service in the same call graph | [llm-budget](https://crates.io/crates/llm-budget) 0.8: same scopes, same wire format, same golden fixtures | |
|
|
72
|
+
|
|
73
|
+
## Stability
|
|
74
|
+
|
|
75
|
+
- The public API is what `agent_budget.__all__` and
|
|
76
|
+
`agent_budget.a2a_plugin.__all__` list. Names starting with `_` are
|
|
77
|
+
private.
|
|
78
|
+
- While the version is 0.x, a breaking change bumps the minor version (0.1
|
|
79
|
+
to 0.2) and is listed in [CHANGELOG.md](../CHANGELOG.md). Patch releases do
|
|
80
|
+
not break working code, so `>=0.1.2,<0.2` is a safe pin.
|
|
81
|
+
- The wire format (`BudgetContext` JSON, baggage keys, the A2A extension
|
|
82
|
+
URI) follows the A2A budget extension draft (a2aproject/A2A#2121). A change
|
|
83
|
+
to it is a minor release, made in step with llm-budget.
|