agentpause 0.4.0__tar.gz → 0.5.0__tar.gz

{agentpause-0.4.0/src/agentpause.egg-info → agentpause-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentpause
-Version: 0.4.0
+Version: 0.5.0
 Summary: Predictive scheduling for autonomous LLM agents: suspend gracefully before rate limits, resume without losing work.
 Author: Andrea Lelio Ciampolillo
 License: MIT
@@ -19,6 +19,8 @@ Provides-Extra: litellm
 Requires-Dist: litellm>=1.40; extra == "litellm"
 Provides-Extra: langgraph
 Requires-Dist: langgraph>=0.2; extra == "langgraph"
+Provides-Extra: gpu
+Requires-Dist: nvidia-ml-py>=12; extra == "gpu"
 Dynamic: license-file
 
 # agentpause
@@ -28,7 +30,10 @@ Dynamic: license-file
 
 The core works on any provider — cloud (OpenAI, Anthropic, Groq) or local — because
 it only serializes application-level state. True KV-cache warm start is an optional
-plugin for self-hosted runtimes (llama.cpp, vLLM).
+plugin for self-hosted runtimes (llama.cpp, vLLM). Predictive suspension itself still
+needs a real budget signal to act on — on a cloud provider that's rate-limit headers;
+for a self-hosted server there are none, see "Local mode: what's actually being
+controlled" below for what to wire in instead.
 
 ## Measured results (from the accompanying research)
 
@@ -308,7 +313,60 @@ Install with `pip install -e ".[langgraph]"`; see `examples/langgraph_quickstart
 ## Scaling up: many providers, many agents, smarter estimates (v0.3)
 
 The same predictive idea — read the budget first, act before the error —
-extends in three directions, and they compose:
+extends to routing across providers and sharing one window across agents.
+
+### Plug and play: `AgentFleet`
+
+Combining `BudgetRouter` + `MultiAgentCoordinator` + an estimator by hand is
+six manual pieces for every single call any agent makes: build the router,
+build the coordinator around `router.budget`, register each agent, build a
+separate estimator per agent, compute `estimated`/`sigma` from it yourself,
+call `coordinator.request(...)`, check `.action == "continue"`, call
+`router.backend(...)` yourself, call `coordinator.complete(...)`, call
+`est.record(...)`. `AgentFleet` collapses that to one constructor call plus
+one `.call(...)` per step:
+
+```python
+from agentpause import AgentFleet
+from agentpause.adapters.openai_compat import OpenAICompatAdapter
+from agentpause.adapters.anthropic import AnthropicAdapter
+
+fleet = AgentFleet(
+    [("groq",   OpenAICompatAdapter.for_model("groq/llama-3.1-8b-instant")),
+     ("claude", AnthropicAdapter("claude-haiku-4-5"))],
+    agents=[("researcher", 1), ("summarizer", 0)],   # (agent_id, priority)
+)
+
+decision, reply, used = fleet.call("researcher", messages)
+if decision.action == "continue":
+    ...  # reply/used are already there; the shared pool is already reconciled
+elif decision.action == "wait":
+    ...  # decision.wait_seconds suggests how long
+else:
+    ...  # checkpoint: this agent's own StateStore, same as single-agent usage
+```
+
+What you must provide: the list of `providers` (same shape `BudgetRouter`
+already accepts), and agent ids if you want explicit priorities. What's
+automatic: the router and coordinator built together on the same clock, a
+fresh and ISOLATED estimator per agent (one agent's learned history never
+leaks into another's), and the whole request → call → reconcile → record
+sequence behind one method. What this does **not** add: checkpoint/resume
+for fleets — `Session`'s predictive decision is computed from its own private
+estimator and there's no seam (by design — the core stays dependency-free,
+`MultiAgentCoordinator`/`BudgetRouter` are deliberately separate,
+adapter-level constructs) to redirect it through the coordinator instead. If
+you need per-agent resumability, pair `AgentFleet` with your own `StateStore`
+per `agent_id`, exactly like single-agent usage does elsewhere in this
+README.
+
+Runnable, no key required: `python examples/fleet_facade_quickstart.py`.
+
+### Under the hood: `BudgetRouter` + `MultiAgentCoordinator` + `FeatureEstimator` on their own
+
+Reach for these directly instead of the facade when you want fine control —
+a custom routing `key=`, direct access to `coordinator.arbitrate()` for a
+batch of simultaneous requests, or a hand-built estimator per agent:
 
 ```python
 from agentpause import BudgetRouter, MultiAgentCoordinator, FeatureEstimator
@@ -326,7 +384,9 @@ router = BudgetRouter(
 # 2. Share ONE rate-limit window across a fleet: every granted call
 #    reserves its predicted cost (estimate + k·σ) from the shared pool,
 #    so agents can't overcommit the window together. Contention is
-#    arbitrated by priority, then longest-waiting.
+#    arbitrated by priority, then longest-waiting — but ONLY in arbitrate()'s
+#    batch call; the streaming request()/complete() pair below has no
+#    priority ordering of its own, first-asked is first-served.
 coord = MultiAgentCoordinator(telemetry=router.budget)
 coord.register("researcher", priority=1)
 coord.register("summarizer")
@@ -343,7 +403,8 @@ if d.action == "continue":
 ```
 
 `FeatureEstimator` is a drop-in for the default estimator
-(`PredictiveScheduler(estimator=FeatureEstimator())`): a dependency-free
+(`PredictiveScheduler(estimator=FeatureEstimator())`, or
+`AgentFleet(..., estimator_factory=FeatureEstimator)`): a dependency-free
 ridge regression over features you declare (tool, model, temperature, …)
 that also learns per-step **latency**, feeding the optional time budget
 (`PredictiveScheduler(time_budget_s=...)`) — if the predicted step can't
@@ -351,7 +412,8 @@ finish before the deadline, the answer is checkpoint, never wait (time,
 unlike tokens, does not refill).
 
 Runnable demo without any key: `python examples/fleet_quickstart.py`
-(routing switch + predictive WAIT under a shared window, in 30 lines of loop).
+(the same three pieces, composed by hand instead of through the facade, for
+anyone who wants to see or override one piece at a time).
 
 ## Plan before you spend, fork your past, and survive a reboot (v0.4)
 
@@ -399,17 +461,136 @@ machine (KV blobs are intentionally machine-local and never migrate).
 from agentpause.llamacpp_kv import LlamaCppSlots, KVStateStore
 
 kv_store = KVStateStore(StateStore(".agentpause"),
-                        slots=LlamaCppSlots(), base_url="http://127.0.0.1:8080")
+                        slots=LlamaCppSlots(), base_url="http://127.0.0.1:8080",
+                        kv_dir="./kv_cache")   # MUST equal --slot-save-path, see below
 kv_store.save_with_kv(checkpoint)               # KV blob first, logical commit second
 cp, info = kv_store.load_with_kv("research-task")
 print(info)  # {"kv_restored": True, "n_restored": 4096} — or a graceful
              # {"kv_restored": False, "reason": "model_mismatch" | "kv_file_missing"}
 ```
 
+`kv_dir` must be the exact same directory the server was started with via
+`--slot-save-path` — a llama-server does not expose that path anywhere in
+`/props` or `/slots`, so this is the one thing agentpause cannot discover on
+its own and must be told. Getting it wrong used to surface only much later,
+at the next restore, as `reason="kv_file_missing"`; `save_with_kv` now checks
+that the blob actually landed in `kv_dir` right after the save call returns
+and raises a clear `KVError` immediately if it didn't, naming the likely
+`kv_dir`/`--slot-save-path` mismatch — caught on the very first save, not
+discovered sessions later.
+
 Runnable demos without any key or server: `python examples/migrate_fork.py`
 (fork + migration story) and `python examples/kv_llamacpp_demo.py` (KV
 save/restore, model-change degradation, independent forked blobs, migration).
 
+## Local mode: what's actually being controlled
+
+A rate-limit budget on a cloud provider comes from real response headers
+(`x-ratelimit-remaining-tokens` and friends — see the tables above). A
+self-hosted `llama-server` sends none of those headers, because there is no
+traffic limit to respect: nobody is throttling requests-per-minute against a
+process running on your own machine.
+
+That has a direct consequence for what `should_suspend()` does locally.
+**Without explicitly configuring `time_budget_s`, `money_budget`, or one of
+the local adapters below, `should_suspend()` has nothing real to evaluate on
+a self-hosted server and will never fire** — unlike on a cloud provider, the
+library will not stop spontaneously for a resource limit, because nothing
+told it one exists. A hand-picked `fallback_remaining=N` papers over this
+with a number that looks real but never depletes. What keeps working with no
+extra configuration either way: manual `checkpoint()` calls, and everything
+the KV-cache section above describes — a warm start stays exactly as
+valuable whether or not a budget signal is also wired in.
+
+### Plug and play: `LocalResourceBudget`
+
+`agentpause.adapters.local_resources` has five independently useful pieces
+for self-hosted llama.cpp (below). Wiring all five by hand for the common
+case — one server, one GPU or none — is six manual steps, two of which need
+a hand calibration most people won't think to do correctly (a
+`bytes_per_token` for the GPU budget, an `estimated_kv_save_s` guess for the
+time budget). `LocalResourceBudget` reduces that to one constructor call
+plus one `.kv_store(...)` call, and self-calibrates the rest from your first
+real checkpoint save — no synthetic probe, no hand measurement:
+
+```python
+from agentpause import PredictiveScheduler, StateStore
+from agentpause.adapters.local_resources import LocalResourceBudget
+
+# kv_dir is the ONE thing this can't discover on its own: it MUST be the
+# exact same directory the server was started with via --slot-save-path
+# (llama-server --slot-save-path ./kv_cache ... <-> kv_dir="./kv_cache").
+local = LocalResourceBudget(kv_dir="./kv_cache")
+
+sched = PredictiveScheduler(backend=my_llm_call, telemetry=local)
+kv_store = local.kv_store(StateStore(".agentpause"))
+
+# normal use from here: kv_store.save_with_kv(checkpoint) whenever you need
+# to suspend. VRAM, the KV-save time reserve, and the disk-space guard
+# threshold all calibrate themselves from that FIRST real save — nothing
+# to measure by hand, nothing to guess.
+```
+
+What you must provide: `kv_dir` (see above), and `base_url` only if your
+server isn't the default `http://127.0.0.1:8080`. What's automatic: the
+configured context size (read from `GET /props` the instant you construct
+it — an unreachable server fails loudly right here, not three steps into a
+run), whether an NVIDIA GPU is present at all (no GPU is treated as
+completely ordinary, never an error), and — from the first real
+`save_with_kv()` call onward — `bytes_per_token`, the measured KV-save
+duration, and a disk-space guard threshold sized from the real blob you just
+wrote (`max(500 MB, 5× that blob's size)`), unless you asked for a specific
+threshold yourself.
+
+Runnable, no real server or GPU required:
+`python examples/local_resource_budget_quickstart.py`.
+
+### Under the hood: the five pieces on their own
+
+Reach for these directly instead of the facade when you want fine control
+over one axis — a custom `used_field=`, a hand-picked `bytes_per_token`, an
+explicit `estimated_kv_save_s`:
+
+- `LlamaCppContextBudget` — reads the ACTUAL configured context size and the
+  ACTUAL tokens used in a slot's KV cache from a real server (`GET /props` +
+  `GET /slots`); never sets `reset_seconds` (a local context window has
+  nothing that refills it on its own).
+- `GPUMemoryBudget` — reads REAL free VRAM straight from the hardware via
+  NVML (`pynvml`, PyPI package `nvidia-ml-py`); refuses to guess a token
+  count without an explicit `bytes_per_token`.
+- `KVAwareTimeBudget` — wraps another telemetry callable and reserves
+  wall-clock time up front for saving the KV-cache blob before a deadline
+  actually hits.
+- `CompositeLocalBudget` — composes several telemetry callables (e.g. the
+  two above) into one, reporting whichever resource is closest to running
+  out, not an average of them.
+- `estimate_local_price_per_1k_tokens` / `estimate_hourly_cost_from_power` /
+  `price_per_1k_tokens_from_estimator` — derive a real `price_per_1k_tokens`
+  for the scheduler's existing monetary constraint from local throughput and
+  a power/rental cost, instead of a provider invoice.
+
+Two honest caveats, not glossed over (they apply whether you use the facade
+or these pieces directly, since the facade is built on top of them):
+
+- **`LlamaCppContextBudget`'s `used_field`**: CONFIRMED live (2026-07-21,
+  against a real llama-server serving Qwen3-8B-GGUF Q4_K_M) to be
+  `n_prompt_tokens` — not documented in the official llama.cpp server
+  README's own example payload at the time this was written, found by
+  actually diffing a slot's `GET /slots` entry before and after a real
+  completion. The default extractor tries it first, with older guesses kept
+  as a fallback chain for other llama.cpp-ecosystem forks/proxies that may
+  shape this differently. Verify against your own server if you're not
+  running mainline llama.cpp, and pass `used_field=` explicitly if it
+  doesn't match.
+- **`GPUMemoryBudget` is NVIDIA-only, today**: it reads VRAM via NVML.
+  AMD/ROCm GPUs expose a different interface (`rocm-smi`/`amdsmi`) that this
+  module does not implement — bring your own `reader_fn=` on ROCm hardware.
+
+Runnable demo, no real server or GPU required:
+`python examples/local_resources_quickstart.py` (the same five signals,
+composed by hand instead of through the facade, for anyone who wants to see
+or override one piece at a time).
+
 ## Why
 
 Current agent frameworks persist state *reactively*: they checkpoint after a step
@@ -433,6 +614,7 @@ Resource-Aware Predictive Scheduler for Autonomous LLM Agents"*.
 | `StateStore` / `Checkpoint` | atomic logical checkpointing with idempotency keys — works on any provider; `compact()` / `summarize_with()` shrink a suspended checkpoint offline |
 | `BudgetRouter` | predictive multi-provider routing: reads every provider's budget first, routes to the most headroom, cools down 429'd providers |
 | `MultiAgentCoordinator` | one shared rate-limit window across many agents: granted calls reserve their predicted cost; `arbitrate()` resolves contention by priority + fairness |
+| `AgentFleet` | plug-and-play facade over the two rows above plus a per-agent estimator: one constructor call + one `.call(agent_id, messages)` per step replaces the request/call/complete/record dance |
 | `ToolQuota` | client-side sliding window for rate-limited tools that expose no headers |
 | `CircuitBreaker` / `FallbackBackend` | reactive safety nets: fail fast on a broken provider, try the next one in order |
 | `adapters.litellm.LiteLLMAdapter` | backend + telemetry for any LiteLLM-supported provider (headers → budget, stale reading → 1-token ping) |
@@ -444,6 +626,12 @@ Resource-Aware Predictive Scheduler for Autonomous LLM Agents"*.
 | `StateStore.export_bundle()` / `import_bundle()` | the checkpoint as a versioned, json-portable process image: suspend on one machine, resume on another at the exact step |
 | `HumanAttentionBudget` | the person in the loop as a rate-limited `Budget`: N asks per rolling window + manual `available()`/`away(until)` override; composes with `decide()` |
 | `llamacpp_kv.LlamaCppSlots` / `KVStateStore` | optional plugin: TRUE warm start for self-hosted llama.cpp via KV-cache save/restore, transactional, with automatic graceful degradation |
+| `adapters.local_resources.LlamaCppContextBudget` | real local-context `Budget` from a llama.cpp server's actual `--ctx-size` and current KV-cache usage (`/props` + `/slots`); never a rate limit, never a `reset_seconds` |
+| `adapters.local_resources.GPUMemoryBudget` | real free-VRAM `Budget` read from NVML (`pynvml`); NVIDIA-only, needs an explicit `bytes_per_token` to report tokens |
+| `adapters.local_resources.KVAwareTimeBudget` | wraps any telemetry callable and takes over `remaining_seconds` itself, reserving time up front for a KV-cache blob save |
+| `adapters.local_resources.CompositeLocalBudget` | composes several telemetry callables into one, most-restrictive `Budget` — see "Local mode" above |
+| `adapters.local_resources.estimate_local_price_per_1k_tokens` / `estimate_hourly_cost_from_power` / `price_per_1k_tokens_from_estimator` | derive a real `price_per_1k_tokens` for a local deployment from throughput and power/rental cost, instead of a provider invoice |
+| `adapters.local_resources.LocalResourceBudget` | plug-and-play facade over the four rows above plus the KV disk guard: one constructor call + `.kv_store(...)`, self-calibrates VRAM/time/disk numbers from your first real checkpoint save |
 
 ## Roadmap
 
@@ -454,12 +642,13 @@ Resource-Aware Predictive Scheduler for Autonomous LLM Agents"*.
 - [x] LangGraph adapter (interrupt + checkpointer)
 - [x] Direct adapters (OpenAI-compatible, Anthropic with prompt caching)
 - [x] Predictive multi-provider routing (`BudgetRouter`)
-- [x] Shared budget across agents (`MultiAgentCoordinator`)
+- [x] Shared budget across agents (`MultiAgentCoordinator`), with a one-call plug-and-play facade (`AgentFleet`)
 - [x] Feature-based cost & latency estimator (`FeatureEstimator`)
 - [x] Task-completion forecast (`session.forecast()`)
 - [x] Checkpoint fork & cross-machine migration
 - [x] Optional KV-cache plugin for llama.cpp (true warm start), incl. fork+KV
 - [x] Human attention as a rate-limited budget (`HumanAttentionBudget`)
+- [x] Real local-resource budgets for self-hosted llama.cpp: context (`LlamaCppContextBudget`), GPU VRAM (`GPUMemoryBudget`), KV-aware time (`KVAwareTimeBudget`), derived local price, composed via `CompositeLocalBudget`, and a one-call plug-and-play facade (`LocalResourceBudget`) that self-calibrates all of the above from a real checkpoint save
 - [ ] CrewAI / AutoGen / LlamaIndex adapters
 - [ ] KV-cache plugin for vLLM
 

{agentpause-0.4.0 → agentpause-0.5.0}/README.md

@@ -5,7 +5,10 @@
 
 The core works on any provider — cloud (OpenAI, Anthropic, Groq) or local — because
 it only serializes application-level state. True KV-cache warm start is an optional
-plugin for self-hosted runtimes (llama.cpp, vLLM).
+plugin for self-hosted runtimes (llama.cpp, vLLM). Predictive suspension itself still
+needs a real budget signal to act on — on a cloud provider that's rate-limit headers;
+for a self-hosted server there are none, see "Local mode: what's actually being
+controlled" below for what to wire in instead.
 
 ## Measured results (from the accompanying research)
 
@@ -285,7 +288,60 @@ Install with `pip install -e ".[langgraph]"`; see `examples/langgraph_quickstart
 ## Scaling up: many providers, many agents, smarter estimates (v0.3)
 
 The same predictive idea — read the budget first, act before the error —
-extends in three directions, and they compose:
+extends to routing across providers and sharing one window across agents.
+
+### Plug and play: `AgentFleet`
+
+Combining `BudgetRouter` + `MultiAgentCoordinator` + an estimator by hand is
+six manual pieces for every single call any agent makes: build the router,
+build the coordinator around `router.budget`, register each agent, build a
+separate estimator per agent, compute `estimated`/`sigma` from it yourself,
+call `coordinator.request(...)`, check `.action == "continue"`, call
+`router.backend(...)` yourself, call `coordinator.complete(...)`, call
+`est.record(...)`. `AgentFleet` collapses that to one constructor call plus
+one `.call(...)` per step:
+
+```python
+from agentpause import AgentFleet
+from agentpause.adapters.openai_compat import OpenAICompatAdapter
+from agentpause.adapters.anthropic import AnthropicAdapter
+
+fleet = AgentFleet(
+    [("groq",   OpenAICompatAdapter.for_model("groq/llama-3.1-8b-instant")),
+     ("claude", AnthropicAdapter("claude-haiku-4-5"))],
+    agents=[("researcher", 1), ("summarizer", 0)],   # (agent_id, priority)
+)
+
+decision, reply, used = fleet.call("researcher", messages)
+if decision.action == "continue":
+    ...  # reply/used are already there; the shared pool is already reconciled
+elif decision.action == "wait":
+    ...  # decision.wait_seconds suggests how long
+else:
+    ...  # checkpoint: this agent's own StateStore, same as single-agent usage
+```
+
+What you must provide: the list of `providers` (same shape `BudgetRouter`
+already accepts), and agent ids if you want explicit priorities. What's
+automatic: the router and coordinator built together on the same clock, a
+fresh and ISOLATED estimator per agent (one agent's learned history never
+leaks into another's), and the whole request → call → reconcile → record
+sequence behind one method. What this does **not** add: checkpoint/resume
+for fleets — `Session`'s predictive decision is computed from its own private
+estimator and there's no seam (by design — the core stays dependency-free,
+`MultiAgentCoordinator`/`BudgetRouter` are deliberately separate,
+adapter-level constructs) to redirect it through the coordinator instead. If
+you need per-agent resumability, pair `AgentFleet` with your own `StateStore`
+per `agent_id`, exactly like single-agent usage does elsewhere in this
+README.
+
+Runnable, no key required: `python examples/fleet_facade_quickstart.py`.
+
+### Under the hood: `BudgetRouter` + `MultiAgentCoordinator` + `FeatureEstimator` on their own
+
+Reach for these directly instead of the facade when you want fine control —
+a custom routing `key=`, direct access to `coordinator.arbitrate()` for a
+batch of simultaneous requests, or a hand-built estimator per agent:
 
 ```python
 from agentpause import BudgetRouter, MultiAgentCoordinator, FeatureEstimator
@@ -303,7 +359,9 @@ router = BudgetRouter(
 # 2. Share ONE rate-limit window across a fleet: every granted call
 #    reserves its predicted cost (estimate + k·σ) from the shared pool,
 #    so agents can't overcommit the window together. Contention is
-#    arbitrated by priority, then longest-waiting.
+#    arbitrated by priority, then longest-waiting — but ONLY in arbitrate()'s
+#    batch call; the streaming request()/complete() pair below has no
+#    priority ordering of its own, first-asked is first-served.
 coord = MultiAgentCoordinator(telemetry=router.budget)
 coord.register("researcher", priority=1)
 coord.register("summarizer")
@@ -320,7 +378,8 @@ if d.action == "continue":
 ```
 
 `FeatureEstimator` is a drop-in for the default estimator
-(`PredictiveScheduler(estimator=FeatureEstimator())`): a dependency-free
+(`PredictiveScheduler(estimator=FeatureEstimator())`, or
+`AgentFleet(..., estimator_factory=FeatureEstimator)`): a dependency-free
 ridge regression over features you declare (tool, model, temperature, …)
 that also learns per-step **latency**, feeding the optional time budget
 (`PredictiveScheduler(time_budget_s=...)`) — if the predicted step can't
@@ -328,7 +387,8 @@ finish before the deadline, the answer is checkpoint, never wait (time,
 unlike tokens, does not refill).
 
 Runnable demo without any key: `python examples/fleet_quickstart.py`
-(routing switch + predictive WAIT under a shared window, in 30 lines of loop).
+(the same three pieces, composed by hand instead of through the facade, for
+anyone who wants to see or override one piece at a time).
 
 ## Plan before you spend, fork your past, and survive a reboot (v0.4)
 
@@ -376,17 +436,136 @@ machine (KV blobs are intentionally machine-local and never migrate).
 from agentpause.llamacpp_kv import LlamaCppSlots, KVStateStore
 
 kv_store = KVStateStore(StateStore(".agentpause"),
-                        slots=LlamaCppSlots(), base_url="http://127.0.0.1:8080")
+                        slots=LlamaCppSlots(), base_url="http://127.0.0.1:8080",
+                        kv_dir="./kv_cache")   # MUST equal --slot-save-path, see below
 kv_store.save_with_kv(checkpoint)               # KV blob first, logical commit second
 cp, info = kv_store.load_with_kv("research-task")
 print(info)  # {"kv_restored": True, "n_restored": 4096} — or a graceful
              # {"kv_restored": False, "reason": "model_mismatch" | "kv_file_missing"}
 ```
 
+`kv_dir` must be the exact same directory the server was started with via
+`--slot-save-path` — a llama-server does not expose that path anywhere in
+`/props` or `/slots`, so this is the one thing agentpause cannot discover on
+its own and must be told. Getting it wrong used to surface only much later,
+at the next restore, as `reason="kv_file_missing"`; `save_with_kv` now checks
+that the blob actually landed in `kv_dir` right after the save call returns
+and raises a clear `KVError` immediately if it didn't, naming the likely
+`kv_dir`/`--slot-save-path` mismatch — caught on the very first save, not
+discovered sessions later.
+
 Runnable demos without any key or server: `python examples/migrate_fork.py`
 (fork + migration story) and `python examples/kv_llamacpp_demo.py` (KV
 save/restore, model-change degradation, independent forked blobs, migration).
 
+## Local mode: what's actually being controlled
+
+A rate-limit budget on a cloud provider comes from real response headers
+(`x-ratelimit-remaining-tokens` and friends — see the tables above). A
+self-hosted `llama-server` sends none of those headers, because there is no
+traffic limit to respect: nobody is throttling requests-per-minute against a
+process running on your own machine.
+
+That has a direct consequence for what `should_suspend()` does locally.
+**Without explicitly configuring `time_budget_s`, `money_budget`, or one of
+the local adapters below, `should_suspend()` has nothing real to evaluate on
+a self-hosted server and will never fire** — unlike on a cloud provider, the
+library will not stop spontaneously for a resource limit, because nothing
+told it one exists. A hand-picked `fallback_remaining=N` papers over this
+with a number that looks real but never depletes. What keeps working with no
+extra configuration either way: manual `checkpoint()` calls, and everything
+the KV-cache section above describes — a warm start stays exactly as
+valuable whether or not a budget signal is also wired in.
+
+### Plug and play: `LocalResourceBudget`
+
+`agentpause.adapters.local_resources` has five independently useful pieces
+for self-hosted llama.cpp (below). Wiring all five by hand for the common
+case — one server, one GPU or none — is six manual steps, two of which need
+a hand calibration most people won't think to do correctly (a
+`bytes_per_token` for the GPU budget, an `estimated_kv_save_s` guess for the
+time budget). `LocalResourceBudget` reduces that to one constructor call
+plus one `.kv_store(...)` call, and self-calibrates the rest from your first
+real checkpoint save — no synthetic probe, no hand measurement:
+
+```python
+from agentpause import PredictiveScheduler, StateStore
+from agentpause.adapters.local_resources import LocalResourceBudget
+
+# kv_dir is the ONE thing this can't discover on its own: it MUST be the
+# exact same directory the server was started with via --slot-save-path
+# (llama-server --slot-save-path ./kv_cache ... <-> kv_dir="./kv_cache").
+local = LocalResourceBudget(kv_dir="./kv_cache")
+
+sched = PredictiveScheduler(backend=my_llm_call, telemetry=local)
+kv_store = local.kv_store(StateStore(".agentpause"))
+
+# normal use from here: kv_store.save_with_kv(checkpoint) whenever you need
+# to suspend. VRAM, the KV-save time reserve, and the disk-space guard
+# threshold all calibrate themselves from that FIRST real save — nothing
+# to measure by hand, nothing to guess.
+```
+
+What you must provide: `kv_dir` (see above), and `base_url` only if your
+server isn't the default `http://127.0.0.1:8080`. What's automatic: the
+configured context size (read from `GET /props` the instant you construct
+it — an unreachable server fails loudly right here, not three steps into a
+run), whether an NVIDIA GPU is present at all (no GPU is treated as
+completely ordinary, never an error), and — from the first real
+`save_with_kv()` call onward — `bytes_per_token`, the measured KV-save
+duration, and a disk-space guard threshold sized from the real blob you just
+wrote (`max(500 MB, 5× that blob's size)`), unless you asked for a specific
+threshold yourself.
+
+Runnable, no real server or GPU required:
+`python examples/local_resource_budget_quickstart.py`.
+
+### Under the hood: the five pieces on their own
+
+Reach for these directly instead of the facade when you want fine control
+over one axis — a custom `used_field=`, a hand-picked `bytes_per_token`, an
+explicit `estimated_kv_save_s`:
+
+- `LlamaCppContextBudget` — reads the ACTUAL configured context size and the
+  ACTUAL tokens used in a slot's KV cache from a real server (`GET /props` +
+  `GET /slots`); never sets `reset_seconds` (a local context window has
+  nothing that refills it on its own).
+- `GPUMemoryBudget` — reads REAL free VRAM straight from the hardware via
+  NVML (`pynvml`, PyPI package `nvidia-ml-py`); refuses to guess a token
+  count without an explicit `bytes_per_token`.
+- `KVAwareTimeBudget` — wraps another telemetry callable and reserves
+  wall-clock time up front for saving the KV-cache blob before a deadline
+  actually hits.
+- `CompositeLocalBudget` — composes several telemetry callables (e.g. the
+  two above) into one, reporting whichever resource is closest to running
+  out, not an average of them.
+- `estimate_local_price_per_1k_tokens` / `estimate_hourly_cost_from_power` /
+  `price_per_1k_tokens_from_estimator` — derive a real `price_per_1k_tokens`
+  for the scheduler's existing monetary constraint from local throughput and
+  a power/rental cost, instead of a provider invoice.
+
+Two honest caveats, not glossed over (they apply whether you use the facade
+or these pieces directly, since the facade is built on top of them):
+
+- **`LlamaCppContextBudget`'s `used_field`**: CONFIRMED live (2026-07-21,
+  against a real llama-server serving Qwen3-8B-GGUF Q4_K_M) to be
+  `n_prompt_tokens` — not documented in the official llama.cpp server
+  README's own example payload at the time this was written, found by
+  actually diffing a slot's `GET /slots` entry before and after a real
+  completion. The default extractor tries it first, with older guesses kept
+  as a fallback chain for other llama.cpp-ecosystem forks/proxies that may
+  shape this differently. Verify against your own server if you're not
+  running mainline llama.cpp, and pass `used_field=` explicitly if it
+  doesn't match.
+- **`GPUMemoryBudget` is NVIDIA-only, today**: it reads VRAM via NVML.
+  AMD/ROCm GPUs expose a different interface (`rocm-smi`/`amdsmi`) that this
+  module does not implement — bring your own `reader_fn=` on ROCm hardware.
+
+Runnable demo, no real server or GPU required:
+`python examples/local_resources_quickstart.py` (the same five signals,
+composed by hand instead of through the facade, for anyone who wants to see
+or override one piece at a time).
+
 ## Why
 
 Current agent frameworks persist state *reactively*: they checkpoint after a step
@@ -410,6 +589,7 @@ Resource-Aware Predictive Scheduler for Autonomous LLM Agents"*.
 | `StateStore` / `Checkpoint` | atomic logical checkpointing with idempotency keys — works on any provider; `compact()` / `summarize_with()` shrink a suspended checkpoint offline |
 | `BudgetRouter` | predictive multi-provider routing: reads every provider's budget first, routes to the most headroom, cools down 429'd providers |
 | `MultiAgentCoordinator` | one shared rate-limit window across many agents: granted calls reserve their predicted cost; `arbitrate()` resolves contention by priority + fairness |
+| `AgentFleet` | plug-and-play facade over the two rows above plus a per-agent estimator: one constructor call + one `.call(agent_id, messages)` per step replaces the request/call/complete/record dance |
 | `ToolQuota` | client-side sliding window for rate-limited tools that expose no headers |
 | `CircuitBreaker` / `FallbackBackend` | reactive safety nets: fail fast on a broken provider, try the next one in order |
 | `adapters.litellm.LiteLLMAdapter` | backend + telemetry for any LiteLLM-supported provider (headers → budget, stale reading → 1-token ping) |
@@ -421,6 +601,12 @@ Resource-Aware Predictive Scheduler for Autonomous LLM Agents"*.
 | `StateStore.export_bundle()` / `import_bundle()` | the checkpoint as a versioned, json-portable process image: suspend on one machine, resume on another at the exact step |
 | `HumanAttentionBudget` | the person in the loop as a rate-limited `Budget`: N asks per rolling window + manual `available()`/`away(until)` override; composes with `decide()` |
 | `llamacpp_kv.LlamaCppSlots` / `KVStateStore` | optional plugin: TRUE warm start for self-hosted llama.cpp via KV-cache save/restore, transactional, with automatic graceful degradation |
+| `adapters.local_resources.LlamaCppContextBudget` | real local-context `Budget` from a llama.cpp server's actual `--ctx-size` and current KV-cache usage (`/props` + `/slots`); never a rate limit, never a `reset_seconds` |
+| `adapters.local_resources.GPUMemoryBudget` | real free-VRAM `Budget` read from NVML (`pynvml`); NVIDIA-only, needs an explicit `bytes_per_token` to report tokens |
+| `adapters.local_resources.KVAwareTimeBudget` | wraps any telemetry callable and takes over `remaining_seconds` itself, reserving time up front for a KV-cache blob save |
+| `adapters.local_resources.CompositeLocalBudget` | composes several telemetry callables into one, most-restrictive `Budget` — see "Local mode" above |
+| `adapters.local_resources.estimate_local_price_per_1k_tokens` / `estimate_hourly_cost_from_power` / `price_per_1k_tokens_from_estimator` | derive a real `price_per_1k_tokens` for a local deployment from throughput and power/rental cost, instead of a provider invoice |
+| `adapters.local_resources.LocalResourceBudget` | plug-and-play facade over the four rows above plus the KV disk guard: one constructor call + `.kv_store(...)`, self-calibrates VRAM/time/disk numbers from your first real checkpoint save |
 
 ## Roadmap
 
@@ -431,12 +617,13 @@ Resource-Aware Predictive Scheduler for Autonomous LLM Agents"*.
 - [x] LangGraph adapter (interrupt + checkpointer)
 - [x] Direct adapters (OpenAI-compatible, Anthropic with prompt caching)
 - [x] Predictive multi-provider routing (`BudgetRouter`)
-- [x] Shared budget across agents (`MultiAgentCoordinator`)
+- [x] Shared budget across agents (`MultiAgentCoordinator`), with a one-call plug-and-play facade (`AgentFleet`)
 - [x] Feature-based cost & latency estimator (`FeatureEstimator`)
 - [x] Task-completion forecast (`session.forecast()`)
 - [x] Checkpoint fork & cross-machine migration
 - [x] Optional KV-cache plugin for llama.cpp (true warm start), incl. fork+KV
 - [x] Human attention as a rate-limited budget (`HumanAttentionBudget`)
+- [x] Real local-resource budgets for self-hosted llama.cpp: context (`LlamaCppContextBudget`), GPU VRAM (`GPUMemoryBudget`), KV-aware time (`KVAwareTimeBudget`), derived local price, composed via `CompositeLocalBudget`, and a one-call plug-and-play facade (`LocalResourceBudget`) that self-calibrates all of the above from a real checkpoint save
 - [ ] CrewAI / AutoGen / LlamaIndex adapters
 - [ ] KV-cache plugin for vLLM
 

{agentpause-0.4.0 → agentpause-0.5.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "agentpause"
-version = "0.4.0"
+version = "0.5.0"
 description = "Predictive scheduling for autonomous LLM agents: suspend gracefully before rate limits, resume without losing work."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -24,6 +24,7 @@ dependencies = []
 dev = ["pytest>=7.0"]
 litellm = ["litellm>=1.40"]
 langgraph = ["langgraph>=0.2"]
+gpu = ["nvidia-ml-py>=12"]
 
 [tool.setuptools.packages.find]
 where = ["src"]

{agentpause-0.4.0 → agentpause-0.5.0}/src/agentpause/__init__.py

@@ -25,11 +25,12 @@ from .retry import RetryPolicy
 from .risk import Budget, Decision, RiskModel, decide, should_checkpoint
 from .router import BudgetRouter
 from .coordinator import MultiAgentCoordinator
+from .fleet import AgentFleet
 from .state import Checkpoint, StateStore
 from .tools import ToolQuota
 from .scheduler import PredictiveScheduler, Session
 
-__version__ = "0.4.0"
+__version__ = "0.5.0"
 
 __all__ = [
     "PredictiveScheduler",
@@ -49,6 +50,7 @@ __all__ = [
     "FallbackBackend",
     "BudgetRouter",
     "MultiAgentCoordinator",
+    "AgentFleet",
     "CircuitBreaker",
     "CircuitOpenError",
     "RegimeDetector",