tool-throttle 0.1.0__tar.gz

tool_throttle-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 hao li
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tool_throttle-0.1.0/PKG-INFO

@@ -0,0 +1,132 @@
+Metadata-Version: 2.4
+Name: tool-throttle
+Version: 0.1.0
+Summary: MCP tool-call throttling: token-bucket rate limits per tool name, a fuse against runaway agents
+Author: hao li
+License: MIT
+Project-URL: Homepage, https://github.com/hahahahahahahahah6/tool-throttle
+Keywords: mcp,ai-agents,rate-limit,throttling,token-bucket
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# tool-throttle
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue.svg)](pyproject.toml)
+[![No dependencies](https://img.shields.io/badge/dependencies-zero-brightgreen.svg)](pyproject.toml)
+[![PyPI](https://img.shields.io/pypi/v/tool-throttle.svg)](https://pypi.org/project/tool-throttle/)
+
+A token-bucket rate limiter for **MCP tool calls** — a fuse against runaway
+agents. When an agent gets stuck in a loop hammering `web_search` 400 times a
+minute, tool-throttle slows it down or stops it, per tool name, with per-tool
+quotas you configure in JSON.
+
+Zero dependencies. Standard library only. Thread-safe.
+
+```bash
+pip install tool-throttle
+```
+
+```python
+from tool_throttle import Throttler, ThrottlerConfig, Quota, REJECT
+
+th = Throttler(ThrottlerConfig(
+    default=Quota(calls=60, per_seconds=60),      # 60 calls/min for everything
+    per_tool={
+        "web_search":  Quota(calls=10, per_seconds=60),  # tight leash
+        "expensive_*": Quota(calls=2, per_seconds=60),   # glob patterns work
+    },
+    on_exceed=REJECT,   # or "queue": wait for a token, up to max_queue_wait
+))
+
+# wrap any callable -- your MCP tool dispatcher, a client method, anything:
+safe_search = th.wrap("web_search", mcp_client.call_tool)
+safe_search("web_search", {"query": "..."})   # -> ThrottledError past quota
+
+# or decide explicitly without consuming a token:
+d = th.decide("web_search")
+# Decision(tool='web_search', allowed=False, wait_seconds=42.1, ...)
+
+th.stats()  # per-tool allowed / rejected / queued / waited_seconds
+```
+
+Config round-trips through JSON, so quotas can live in a file:
+
+```python
+cfg = ThrottlerConfig(
+    default=Quota(calls=60, per_seconds=60),
+    per_tool={"web_*": Quota(calls=10, per_seconds=60, burst=3)},
+    on_exceed="queue",
+    max_queue_wait=30.0,
+)
+cfg.save("throttle.json")
+```
+
+## CLI
+
+```bash
+# dry-run: print a decision per {"tool": ...} JSON line on stdin
+echo '{"tool": "web_search"}' | tool-throttle check --config throttle.json
+
+# enforce: consume tokens (queues or rejects over-quota calls)
+echo '{"tool": "web_search"}' | tool-throttle acquire --config throttle.json
+
+# built-in demo of a looping agent being fused
+tool-throttle demo
+```
+
+## How it works
+
+Each distinct tool name gets its own `TokenBucket`, sized by the first
+matching quota in `per_tool` (glob patterns, insertion order) or `default`.
+Buckets refill steadily over time, so a `Quota(calls=10, per_seconds=60)`
+allows 10 calls per minute sustained, with a burst of up to `burst`
+(default: `calls`).
+
+On quota exhaustion:
+- `on_exceed="reject"` (default in the demo): raise `ThrottledError` with a
+  `retry_after` hint — fail fast.
+- `on_exceed="queue"`: block until a token frees up, bounded by
+  `max_queue_wait` seconds; if the wait would exceed the bound, raise
+  `ThrottledError(reason="queue_wait_exceeded")`.
+
+## Differentiation: why not just rate-limit at the API gateway?
+
+API gateways throttle **HTTP requests**. That is the wrong layer for agent
+safety:
+
+- One HTTP request can fan out into many tool calls; one tool call pattern
+  can span many HTTP endpoints. Gateway counters never see *tool-call
+  semantics* — they see bytes and paths.
+- Gateway limits are per API key / per IP. An agent looping on a *single
+  expensive tool* (say, a search tool that costs $0.02 per call) looks
+  perfectly normal at the HTTP layer while burning money at the tool layer.
+- Quotas here are expressed in the language of the agent: *tool names and
+  glob patterns* (`expensive_*`), with per-tool bursts and queue-or-reject
+  policies — the thing an agent developer actually wants to tune.
+
+tool-throttle is the **fuse at the MCP tool-call semantic layer**: it sits
+between the agent's tool-call intent and execution, trips before the loop
+does real damage, and reports per-tool stats so you can see which tool the
+agent was abusing. It complements gateway rate limiting; it does not
+replace it.
+
+## API reference
+
+| Symbol | Description |
+|---|---|
+| `Throttler(config)` | the interceptor; one per agent/process |
+| `ThrottlerConfig(default, per_tool, on_exceed, max_queue_wait)` | quotas + policy |
+| `Quota(calls, per_seconds=60, burst=None)` | rate quota for one pattern |
+| `th.acquire(tool)` | consume a token; queues or raises `ThrottledError` |
+| `th.decide(tool)` | dry-run check, does not consume |
+| `th.wrap(tool, fn)` / `th.call(tool, fn, *a, **k)` | throttle a callable |
+| `th.stats()` / `th.reset(tool?)` / `th.clear()` | observability and reset |
+| `ThrottledError` | `.tool`, `.retry_after`, `.reason` |
+| `TokenBucket(capacity, refill_per_second, clock=...)` | the primitive underneath |
+
+## License
+
+MIT — see [LICENSE](LICENSE).

tool_throttle-0.1.0/README.md

@@ -0,0 +1,119 @@
+# tool-throttle
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue.svg)](pyproject.toml)
+[![No dependencies](https://img.shields.io/badge/dependencies-zero-brightgreen.svg)](pyproject.toml)
+[![PyPI](https://img.shields.io/pypi/v/tool-throttle.svg)](https://pypi.org/project/tool-throttle/)
+
+A token-bucket rate limiter for **MCP tool calls** — a fuse against runaway
+agents. When an agent gets stuck in a loop hammering `web_search` 400 times a
+minute, tool-throttle slows it down or stops it, per tool name, with per-tool
+quotas you configure in JSON.
+
+Zero dependencies. Standard library only. Thread-safe.
+
+```bash
+pip install tool-throttle
+```
+
+```python
+from tool_throttle import Throttler, ThrottlerConfig, Quota, REJECT
+
+th = Throttler(ThrottlerConfig(
+    default=Quota(calls=60, per_seconds=60),      # 60 calls/min for everything
+    per_tool={
+        "web_search":  Quota(calls=10, per_seconds=60),  # tight leash
+        "expensive_*": Quota(calls=2, per_seconds=60),   # glob patterns work
+    },
+    on_exceed=REJECT,   # or "queue": wait for a token, up to max_queue_wait
+))
+
+# wrap any callable -- your MCP tool dispatcher, a client method, anything:
+safe_search = th.wrap("web_search", mcp_client.call_tool)
+safe_search("web_search", {"query": "..."})   # -> ThrottledError past quota
+
+# or decide explicitly without consuming a token:
+d = th.decide("web_search")
+# Decision(tool='web_search', allowed=False, wait_seconds=42.1, ...)
+
+th.stats()  # per-tool allowed / rejected / queued / waited_seconds
+```
+
+Config round-trips through JSON, so quotas can live in a file:
+
+```python
+cfg = ThrottlerConfig(
+    default=Quota(calls=60, per_seconds=60),
+    per_tool={"web_*": Quota(calls=10, per_seconds=60, burst=3)},
+    on_exceed="queue",
+    max_queue_wait=30.0,
+)
+cfg.save("throttle.json")
+```
+
+## CLI
+
+```bash
+# dry-run: print a decision per {"tool": ...} JSON line on stdin
+echo '{"tool": "web_search"}' | tool-throttle check --config throttle.json
+
+# enforce: consume tokens (queues or rejects over-quota calls)
+echo '{"tool": "web_search"}' | tool-throttle acquire --config throttle.json
+
+# built-in demo of a looping agent being fused
+tool-throttle demo
+```
+
+## How it works
+
+Each distinct tool name gets its own `TokenBucket`, sized by the first
+matching quota in `per_tool` (glob patterns, insertion order) or `default`.
+Buckets refill steadily over time, so a `Quota(calls=10, per_seconds=60)`
+allows 10 calls per minute sustained, with a burst of up to `burst`
+(default: `calls`).
+
+On quota exhaustion:
+- `on_exceed="reject"` (default in the demo): raise `ThrottledError` with a
+  `retry_after` hint — fail fast.
+- `on_exceed="queue"`: block until a token frees up, bounded by
+  `max_queue_wait` seconds; if the wait would exceed the bound, raise
+  `ThrottledError(reason="queue_wait_exceeded")`.
+
+## Differentiation: why not just rate-limit at the API gateway?
+
+API gateways throttle **HTTP requests**. That is the wrong layer for agent
+safety:
+
+- One HTTP request can fan out into many tool calls; one tool call pattern
+  can span many HTTP endpoints. Gateway counters never see *tool-call
+  semantics* — they see bytes and paths.
+- Gateway limits are per API key / per IP. An agent looping on a *single
+  expensive tool* (say, a search tool that costs $0.02 per call) looks
+  perfectly normal at the HTTP layer while burning money at the tool layer.
+- Quotas here are expressed in the language of the agent: *tool names and
+  glob patterns* (`expensive_*`), with per-tool bursts and queue-or-reject
+  policies — the thing an agent developer actually wants to tune.
+
+tool-throttle is the **fuse at the MCP tool-call semantic layer**: it sits
+between the agent's tool-call intent and execution, trips before the loop
+does real damage, and reports per-tool stats so you can see which tool the
+agent was abusing. It complements gateway rate limiting; it does not
+replace it.
+
+## API reference
+
+| Symbol | Description |
+|---|---|
+| `Throttler(config)` | the interceptor; one per agent/process |
+| `ThrottlerConfig(default, per_tool, on_exceed, max_queue_wait)` | quotas + policy |
+| `Quota(calls, per_seconds=60, burst=None)` | rate quota for one pattern |
+| `th.acquire(tool)` | consume a token; queues or raises `ThrottledError` |
+| `th.decide(tool)` | dry-run check, does not consume |
+| `th.wrap(tool, fn)` / `th.call(tool, fn, *a, **k)` | throttle a callable |
+| `th.stats()` / `th.reset(tool?)` / `th.clear()` | observability and reset |
+| `ThrottledError` | `.tool`, `.retry_after`, `.reason` |
+| `TokenBucket(capacity, refill_per_second, clock=...)` | the primitive underneath |
+
+## License
+
+MIT — see [LICENSE](LICENSE).

tool_throttle-0.1.0/pyproject.toml

@@ -0,0 +1,22 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "tool-throttle"
+version = "0.1.0"
+description = "MCP tool-call throttling: token-bucket rate limits per tool name, a fuse against runaway agents"
+readme = "README.md"
+requires-python = ">=3.9"
+license = {text = "MIT"}
+authors = [{name = "hao li"}]
+keywords = ["mcp", "ai-agents", "rate-limit", "throttling", "token-bucket"]
+
+[project.scripts]
+tool-throttle = "tool_throttle.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/hahahahahahahahah6/tool-throttle"
+
+[tool.setuptools.packages.find]
+include = ["tool_throttle*"]

tool_throttle-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

tool_throttle-0.1.0/tests/test_smoke.py

@@ -0,0 +1,188 @@
+"""Smoke tests for tool-throttle. All must be green before shipping."""
+
+import json
+import os
+import tempfile
+import time
+
+import pytest
+
+from tool_throttle import (
+    QUEUE, REJECT, Quota, ThrottledError, Throttler, ThrottlerConfig,
+    TokenBucket,
+)
+
+
+class FakeClock:
+    def __init__(self):
+        self.t = 1000.0
+
+    def __call__(self):
+        return self.t
+
+    def advance(self, dt):
+        self.t += dt
+
+
+# 1. tokens refill at the configured rate (deterministic, fake clock)
+def test_refill_math():
+    c = FakeClock()
+    b = TokenBucket(capacity=5, refill_per_second=2.0, clock=c)
+    assert b.try_consume(5) is True
+    assert b.try_consume(1) is False
+    c.advance(1.0)
+    assert b.peek() == pytest.approx(2.0)
+    assert b.try_consume(2) is True
+    assert b.try_consume(1) is False
+
+
+# 2. idle buckets never exceed capacity (burst cap)
+def test_burst_cap():
+    c = FakeClock()
+    b = TokenBucket(capacity=3, refill_per_second=100.0, clock=c)
+    b.try_consume(3)
+    c.advance(3600)  # idle for an hour
+    assert b.peek() == pytest.approx(3.0)
+
+
+# 3. per-tool quotas override the default; unmatched tools use default
+def test_per_tool_override():
+    c = FakeClock()
+    th = Throttler(ThrottlerConfig(
+        default=Quota(calls=60, per_seconds=60),
+        per_tool={"expensive_tool": Quota(calls=2, per_seconds=60)},
+        on_exceed=REJECT), clock=c)
+    th.acquire("expensive_tool")
+    th.acquire("expensive_tool")
+    with pytest.raises(ThrottledError):
+        th.acquire("expensive_tool")
+    # default tool is unaffected
+    th.acquire("cheap_tool")
+    d = th.decide("cheap_tool")
+    assert d.pattern is None  # default has no pattern
+
+
+# 4. first matching pattern wins (insertion order)
+def test_first_match_wins():
+    c = FakeClock()
+    th = Throttler(ThrottlerConfig(
+        per_tool={"web_*": Quota(calls=2, per_seconds=60),
+                  "web_search": Quota(calls=50, per_seconds=60)},
+        on_exceed=REJECT), clock=c)
+    d = th.decide("web_search")
+    assert d.pattern == "web_*"
+    th.acquire("web_search")
+    th.acquire("web_search")
+    with pytest.raises(ThrottledError):
+        th.acquire("web_search")
+
+
+# 5. reject policy fails fast and reports retry_after
+def test_reject_reports_retry_after():
+    c = FakeClock()
+    th = Throttler(ThrottlerConfig(
+        default=Quota(calls=1, per_seconds=60), on_exceed=REJECT), clock=c)
+    th.acquire("t")
+    t0 = time.monotonic()
+    with pytest.raises(ThrottledError) as ei:
+        th.acquire("t")
+    assert time.monotonic() - t0 < 0.5  # fast, no blocking
+    assert ei.value.retry_after == pytest.approx(60.0, abs=1.0)
+    assert ei.value.tool == "t"
+    assert th.stats()["t"]["rejected"] == 1
+
+
+# 6. queue policy waits for a token then proceeds (real clock)
+def test_queue_waits_then_proceeds():
+    th = Throttler(ThrottlerConfig(
+        default=Quota(calls=2, per_seconds=1, burst=1),  # 1 token every 0.5s
+        on_exceed=QUEUE, max_queue_wait=5.0))
+    th.acquire("t")  # consumes the 1-token burst
+    t0 = time.monotonic()
+    d = th.acquire("t")
+    elapsed = time.monotonic() - t0
+    assert elapsed >= 0.4  # actually waited for refill
+    assert d.allowed and d.wait_seconds >= 0.4
+    assert th.stats()["t"]["queued"] == 1
+
+
+# 7. queue policy is bounded: a call needing an hour-long wait fails fast
+def test_queue_bound_never_blocks():
+    th = Throttler(ThrottlerConfig(
+        default=Quota(calls=1, per_seconds=3600),  # ~1/hour refill
+        on_exceed=QUEUE, max_queue_wait=0.2))
+    th.acquire("t")
+    t0 = time.monotonic()
+    with pytest.raises(ThrottledError) as ei:
+        th.acquire("t")
+    elapsed = time.monotonic() - t0
+    assert elapsed < 1.0  # bounded, never blocks for the full hour
+    assert ei.value.reason == "quota_exceeded"
+    assert ei.value.retry_after > 3000
+    assert th.stats()["t"]["rejected"] == 1
+
+
+# 8. wrap() passes return values through and enforces the quota
+def test_wrap_integration():
+    c = FakeClock()
+    th = Throttler(ThrottlerConfig(
+        default=Quota(calls=3, per_seconds=60), on_exceed=REJECT), clock=c)
+
+    def add(a, b):
+        """adds two numbers"""
+        return a + b
+
+    safe_add = th.wrap("math", add)
+    assert safe_add(2, 3) == 5
+    assert safe_add.__name__ == "add"
+    safe_add(1, 1)
+    safe_add(1, 1)
+    with pytest.raises(ThrottledError):
+        safe_add(1, 1)
+    s = th.stats()["math"]
+    assert s["allowed"] == 3 and s["rejected"] == 1
+
+
+# 9. stats() exposes per-tool counters
+def test_stats_shape():
+    c = FakeClock()
+    th = Throttler(ThrottlerConfig(
+        default=Quota(calls=1, per_seconds=60),
+        per_tool={"web_*": Quota(calls=5, per_seconds=60)},
+        on_exceed=REJECT), clock=c)
+    th.acquire("web_search")
+    try:
+        th.acquire("other")
+        th.acquire("other")
+    except ThrottledError:
+        pass
+    s = th.stats()
+    assert set(s) == {"web_search", "other"}
+    for tool, counters in s.items():
+        assert set(counters) == {"allowed", "rejected", "queued",
+                                 "waited_seconds", "quota", "pattern"}
+    assert s["web_search"]["allowed"] == 1
+    assert s["other"]["rejected"] == 1
+    assert s["web_search"]["pattern"] == "web_*"
+
+
+# 10. config survives a JSON round-trip
+def test_config_json_roundtrip():
+    cfg = ThrottlerConfig(
+        default=Quota(calls=30, per_seconds=60),
+        per_tool={"expensive_*": Quota(calls=2, per_seconds=60, burst=4)},
+        on_exceed=REJECT, max_queue_wait=7.5)
+    with tempfile.TemporaryDirectory() as d:
+        p = os.path.join(d, "throttle.json")
+        cfg.save(p)
+        loaded = ThrottlerConfig.load(p)
+    assert loaded == cfg
+    q, pat = loaded.quota_for("expensive_llm")
+    assert (q.calls, q.per_seconds, q.burst, pat) == (2, 60.0, 4, "expensive_*")
+    # round-tripped config behaves identically (burst=4 -> 4 tokens)
+    c = FakeClock()
+    th = Throttler(loaded, clock=c)
+    for _ in range(4):
+        th.acquire("expensive_llm")
+    with pytest.raises(ThrottledError):
+        th.acquire("expensive_llm")

tool_throttle-0.1.0/tool_throttle/__init__.py

@@ -0,0 +1,17 @@
+"""tool-throttle: MCP tool-call throttling (token bucket per tool name)."""
+
+from .bucket import TokenBucket
+from .config import QUEUE, REJECT, Quota, ThrottlerConfig
+from .throttler import Decision, ThrottledError, Throttler
+
+__version__ = "0.1.0"
+__all__ = [
+    "TokenBucket",
+    "Quota",
+    "ThrottlerConfig",
+    "QUEUE",
+    "REJECT",
+    "Decision",
+    "ThrottledError",
+    "Throttler",
+]

tool_throttle-0.1.0/tool_throttle/bucket.py

@@ -0,0 +1,69 @@
+"""Thread-safe token bucket used by tool-throttle.
+
+The bucket is deliberately simple: a fixed ``capacity`` (burst size) and a
+steady ``refill_per_second`` rate. A ``clock`` callable can be injected so
+tests can control time deterministically.
+"""
+
+from __future__ import annotations
+
+import threading
+import time
+
+
+class TokenBucket:
+    """A classic token bucket.
+
+    * ``capacity`` -- maximum burst size (tokens).
+    * ``refill_per_second`` -- steady refill rate (tokens per second).
+    """
+
+    def __init__(self, capacity: float, refill_per_second: float,
+                 clock=time.monotonic) -> None:
+        if capacity <= 0:
+            raise ValueError("capacity must be > 0")
+        if refill_per_second <= 0:
+            raise ValueError("refill_per_second must be > 0")
+        self.capacity = float(capacity)
+        self.refill_per_second = float(refill_per_second)
+        self._clock = clock
+        self._lock = threading.Lock()
+        self._tokens = float(capacity)
+        self._last = float(clock())
+
+    def _refill(self, now: float) -> None:
+        elapsed = now - self._last
+        if elapsed > 0:
+            self._tokens = min(self.capacity,
+                               self._tokens + elapsed * self.refill_per_second)
+            self._last = now
+
+    def try_consume(self, n: float = 1.0) -> bool:
+        """Consume ``n`` tokens if available; return True, else leave state
+        untouched and return False."""
+        with self._lock:
+            self._refill(self._clock())
+            if self._tokens >= n:
+                self._tokens -= n
+                return True
+            return False
+
+    def peek(self) -> float:
+        """Current token count after applying refill (does not consume)."""
+        with self._lock:
+            self._refill(self._clock())
+            return self._tokens
+
+    def deficit_wait(self, n: float = 1.0) -> float:
+        """Seconds until ``n`` tokens are available (>= 0)."""
+        with self._lock:
+            self._refill(self._clock())
+            if self._tokens >= n:
+                return 0.0
+            return (n - self._tokens) / self.refill_per_second
+
+    def reset(self) -> None:
+        """Refill the bucket to full capacity."""
+        with self._lock:
+            self._tokens = self.capacity
+            self._last = float(self._clock())

tool_throttle-0.1.0/tool_throttle/cli.py

@@ -0,0 +1,117 @@
+"""CLI for tool-throttle.
+
+Subcommands:
+  check   read {"tool": ...} JSON lines from stdin, print a throttling
+          decision per line (dry-run: does NOT consume tokens).
+  acquire read {"tool": ...} JSON lines from stdin, apply the throttle
+          (consumes tokens; queues or rejects), print the decision.
+  demo    run a built-in scenario showing queue vs reject behaviour.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+
+from . import __version__
+from .config import QUEUE, REJECT, Quota, ThrottlerConfig
+from .throttler import ThrottledError, Throttler
+
+
+def demo_config() -> ThrottlerConfig:
+    return ThrottlerConfig(
+        default=Quota(calls=60, per_seconds=60),
+        per_tool={
+            "web_search": Quota(calls=10, per_seconds=60),
+            "expensive_*": Quota(calls=2, per_seconds=60),
+        },
+        on_exceed=REJECT,
+        max_queue_wait=5.0,
+    )
+
+
+def _iter_stdin():
+    for line in sys.stdin:
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            yield json.loads(line)
+        except json.JSONDecodeError as e:
+            print(json.dumps({"error": f"bad JSON: {e}"}), flush=True)
+
+
+def cmd_check(args) -> int:
+    cfg = ThrottlerConfig.load(args.config) if args.config else demo_config()
+    th = Throttler(cfg)
+    for obj in _iter_stdin():
+        tool = obj.get("tool", "")
+        d = th.decide(tool)
+        print(json.dumps({"tool": d.tool, "allowed": d.allowed,
+                          "wait_seconds": round(d.wait_seconds, 3),
+                          "reason": d.reason, "pattern": d.pattern}),
+              flush=True)
+    return 0
+
+
+def cmd_acquire(args) -> int:
+    cfg = ThrottlerConfig.load(args.config) if args.config else demo_config()
+    th = Throttler(cfg)
+    for obj in _iter_stdin():
+        tool = obj.get("tool", "")
+        try:
+            d = th.acquire(tool)
+            print(json.dumps({"tool": d.tool, "allowed": True,
+                              "wait_seconds": round(d.wait_seconds, 3),
+                              "reason": d.reason, "pattern": d.pattern}),
+                  flush=True)
+        except ThrottledError as e:
+            print(json.dumps({"tool": tool, "allowed": False,
+                              "wait_seconds": round(e.retry_after, 3),
+                              "reason": e.reason, "pattern": None}),
+                  flush=True)
+    return 0
+
+
+def cmd_demo(_args) -> int:
+    print("== tool-throttle demo: an agent looping web_search ==")
+    cfg = ThrottlerConfig(
+        default=Quota(calls=60, per_seconds=60),
+        per_tool={"web_search": Quota(calls=5, per_seconds=60)},
+        on_exceed=REJECT,
+    )
+    th = Throttler(cfg)
+    allowed, rejected = 0, 0
+    for _ in range(12):
+        try:
+            th.acquire("web_search")
+            allowed += 1
+        except ThrottledError:
+            rejected += 1
+    print(f"web_search: {allowed} allowed, {rejected} rejected "
+          f"(quota 5/min) -- the loop is fused, not looping forever")
+    print(f"stats: {json.dumps(th.stats(), indent=2)}")
+    return 0
+
+
+def main(argv=None) -> int:
+    p = argparse.ArgumentParser(prog="tool-throttle",
+                                description="MCP tool-call throttler")
+    p.add_argument("--version", action="version", version=__version__)
+    sub = p.add_subparsers(dest="cmd", required=True)
+    for name, fn, help_ in (
+            ("check", cmd_check, "dry-run decisions for stdin JSON lines"),
+            ("acquire", cmd_acquire, "consume tokens for stdin JSON lines"),
+            ("demo", cmd_demo, "show throttling on a looping agent")):
+        s = sub.add_parser(name, help=help_)
+        if name != "demo":
+            s.add_argument("--config", default=None,
+                           help="path to a throttle config JSON file")
+        s.set_defaults(func=fn)
+    args = p.parse_args(argv)
+    return args.func(args)
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tool_throttle-0.1.0/tool_throttle/config.py

@@ -0,0 +1,102 @@
+"""Configuration for tool-throttle: per-tool quotas, JSON round-trip."""
+
+from __future__ import annotations
+
+import fnmatch
+import json
+from dataclasses import dataclass, field
+
+
+@dataclass
+class Quota:
+    """Rate quota: ``calls`` tokens per ``per_seconds`` seconds.
+
+    ``burst`` caps the burst size (defaults to ``calls``).
+    """
+    calls: int
+    per_seconds: float = 60.0
+    burst: int | None = None
+
+    def __post_init__(self) -> None:
+        if self.calls <= 0:
+            raise ValueError("calls must be > 0")
+        if self.per_seconds <= 0:
+            raise ValueError("per_seconds must be > 0")
+        if self.burst is not None and self.burst <= 0:
+            raise ValueError("burst must be > 0")
+
+    @property
+    def capacity(self) -> float:
+        return float(self.burst if self.burst is not None else self.calls)
+
+    @property
+    def refill_per_second(self) -> float:
+        return float(self.calls) / float(self.per_seconds)
+
+    def to_dict(self) -> dict:
+        return {"calls": self.calls, "per_seconds": self.per_seconds,
+                "burst": self.burst}
+
+    @classmethod
+    def from_dict(cls, d: dict) -> "Quota":
+        return cls(calls=d["calls"], per_seconds=d.get("per_seconds", 60.0),
+                   burst=d.get("burst"))
+
+
+#: What to do when a call exceeds its quota.
+QUEUE = "queue"    #: block until a token is available (bounded by max_queue_wait)
+REJECT = "reject"  #: fail fast with ThrottledError
+
+
+@dataclass
+class ThrottlerConfig:
+    """Full throttle configuration.
+
+    ``per_tool`` maps glob patterns (e.g. ``"web_*"``) to quotas.
+    Patterns are matched with :func:`fnmatch.fnmatchcase` in insertion
+    order; the first match wins. Tools matching nothing use ``default``.
+    """
+    default: Quota = field(default_factory=lambda: Quota(calls=60, per_seconds=60))
+    per_tool: dict[str, Quota] = field(default_factory=dict)
+    on_exceed: str = QUEUE
+    max_queue_wait: float = 30.0
+
+    def __post_init__(self) -> None:
+        if self.on_exceed not in (QUEUE, REJECT):
+            raise ValueError("on_exceed must be 'queue' or 'reject'")
+        if self.max_queue_wait < 0:
+            raise ValueError("max_queue_wait must be >= 0")
+
+    def quota_for(self, tool: str) -> tuple[Quota, str | None]:
+        """Return (quota, matched_pattern) for a tool name."""
+        for pattern, quota in self.per_tool.items():
+            if fnmatch.fnmatchcase(tool, pattern):
+                return quota, pattern
+        return self.default, None
+
+    def to_dict(self) -> dict:
+        return {
+            "default": self.default.to_dict(),
+            "per_tool": {p: q.to_dict() for p, q in self.per_tool.items()},
+            "on_exceed": self.on_exceed,
+            "max_queue_wait": self.max_queue_wait,
+        }
+
+    @classmethod
+    def from_dict(cls, d: dict) -> "ThrottlerConfig":
+        return cls(
+            default=Quota.from_dict(d.get("default", {"calls": 60})),
+            per_tool={p: Quota.from_dict(q)
+                      for p, q in d.get("per_tool", {}).items()},
+            on_exceed=d.get("on_exceed", QUEUE),
+            max_queue_wait=d.get("max_queue_wait", 30.0),
+        )
+
+    def save(self, path: str) -> None:
+        with open(path, "w") as f:
+            json.dump(self.to_dict(), f, indent=2)
+
+    @classmethod
+    def load(cls, path: str) -> "ThrottlerConfig":
+        with open(path) as f:
+            return cls.from_dict(json.load(f))

tool_throttle-0.1.0/tool_throttle/throttler.py

@@ -0,0 +1,162 @@
+"""The MCP tool-call throttle: intercept tool calls, apply token-bucket
+quotas per tool name, and either queue or reject over-quota calls.
+
+This is the semantic-layer "fuse" for agents: it does not care about HTTP
+or API keys, it cares about *tool call intent* -- e.g. an agent stuck in a
+loop hammering ``web_search`` 400 times a minute.
+"""
+
+from __future__ import annotations
+
+import threading
+import time
+from dataclasses import dataclass
+
+from .bucket import TokenBucket
+from .config import QUEUE, REJECT, Quota, ThrottlerConfig
+
+
+class ThrottledError(Exception):
+    """Raised when a tool call exceeds its quota.
+
+    ``retry_after`` is seconds until the call would be allowed (0 if the
+    caller should not retry, e.g. queue-wait exhausted).
+    """
+
+    def __init__(self, tool: str, retry_after: float, reason: str) -> None:
+        super().__init__(f"tool call throttled: {tool} ({reason})")
+        self.tool = tool
+        self.retry_after = float(retry_after)
+        self.reason = reason
+
+
+@dataclass
+class Decision:
+    """The outcome of a throttling check for one tool call."""
+    tool: str
+    allowed: bool
+    wait_seconds: float       # 0 if allowed immediately
+    reason: str               # "ok" | "quota_exceeded"
+    pattern: str | None = None  # which per_tool pattern matched (if any)
+
+
+class Throttler:
+    """Intercept and throttle tool calls by tool name.
+
+    Each distinct tool name gets its own :class:`TokenBucket` sized by the
+    first matching quota in ``config.per_tool`` (or ``config.default``).
+    """
+
+    def __init__(self, config: ThrottlerConfig | None = None,
+                 clock=time.monotonic) -> None:
+        self.config = config or ThrottlerConfig()
+        self._clock = clock
+        self._lock = threading.Lock()
+        self._cond = threading.Condition(self._lock)
+        self._buckets: dict[str, tuple[TokenBucket, Quota, str | None]] = {}
+        self._stats: dict[str, dict[str, float]] = {}
+
+    # -- internals ------------------------------------------------------
+
+    def _entry(self, tool: str) -> tuple[TokenBucket, Quota, str | None]:
+        entry = self._buckets.get(tool)
+        if entry is None:
+            quota, pattern = self.config.quota_for(tool)
+            entry = (TokenBucket(quota.capacity, quota.refill_per_second,
+                                 clock=self._clock), quota, pattern)
+            self._buckets[tool] = entry
+            self._stats[tool] = {"allowed": 0, "rejected": 0, "queued": 0,
+                                 "waited_seconds": 0.0,
+                                 "quota": f"{quota.calls}/{quota.per_seconds}s",
+                                 "pattern": pattern or "default"}
+        return entry
+
+    # -- public API -----------------------------------------------------
+
+    def decide(self, tool: str, n: float = 1.0) -> Decision:
+        """Check whether a call would be allowed *without* consuming tokens."""
+        with self._lock:
+            bucket, _, pattern = self._entry(tool)
+        wait = bucket.deficit_wait(n)
+        if wait <= 0:
+            return Decision(tool, True, 0.0, "ok", pattern)
+        return Decision(tool, False, wait, "quota_exceeded", pattern)
+
+    def acquire(self, tool: str, n: float = 1.0) -> Decision:
+        """Consume a token for ``tool``.
+
+        With ``on_exceed="queue"`` this blocks (up to ``max_queue_wait``
+        seconds) until a token is available, then proceeds. With
+        ``on_exceed="reject"`` -- or when the wait would exceed the bound --
+        raises :class:`ThrottledError`.
+        """
+        with self._lock:
+            bucket, quota, pattern = self._entry(tool)
+            stats = self._stats[tool]
+        if bucket.try_consume(n):
+            with self._lock:
+                stats["allowed"] += 1
+            return Decision(tool, True, 0.0, "ok", pattern)
+
+        wait_needed = bucket.deficit_wait(n)
+        remaining = self.config.max_queue_wait
+        if self.config.on_exceed == REJECT or wait_needed > remaining:
+            with self._lock:
+                stats["rejected"] += 1
+            raise ThrottledError(tool, wait_needed, "quota_exceeded")
+
+        # queue: wait in small slices until a token frees up or we time out
+        queued = True
+        start = self._clock()
+        deadline = start + remaining
+        while True:
+            sleep_for = min(wait_needed, 0.05)
+            with self._cond:
+                self._cond.wait(timeout=sleep_for)
+            if bucket.try_consume(n):
+                waited = max(0.0, self._clock() - start)
+                with self._lock:
+                    stats["allowed"] += 1
+                    if queued:
+                        stats["queued"] += 1
+                        stats["waited_seconds"] += waited
+                return Decision(tool, True, waited, "queued", pattern)
+            wait_needed = bucket.deficit_wait(n)
+            remaining = deadline - self._clock()
+            if remaining <= 0:
+                with self._lock:
+                    stats["rejected"] += 1
+                raise ThrottledError(tool, wait_needed, "queue_wait_exceeded")
+
+    def wrap(self, tool: str, fn):
+        """Return a wrapper that throttles calls to ``fn`` as ``tool``."""
+        def _wrapped(*args, **kwargs):
+            self.acquire(tool)
+            return fn(*args, **kwargs)
+        _wrapped.__name__ = getattr(fn, "__name__", f"throttled_{tool}")
+        _wrapped.__doc__ = getattr(fn, "__doc__", None)
+        return _wrapped
+
+    def call(self, tool: str, fn, *args, **kwargs):
+        """Call ``fn(*args, **kwargs)`` through the throttle as ``tool``."""
+        self.acquire(tool)
+        return fn(*args, **kwargs)
+
+    def stats(self) -> dict[str, dict[str, float]]:
+        """Per-tool counters: allowed / rejected / queued / waited_seconds."""
+        with self._lock:
+            return {t: dict(s) for t, s in self._stats.items()}
+
+    def reset(self, tool: str | None = None) -> None:
+        """Refill bucket(s) to full. ``tool=None`` resets everything."""
+        with self._lock:
+            targets = [tool] if tool else list(self._buckets)
+            for t in targets:
+                if t in self._buckets:
+                    self._buckets[t][0].reset()
+
+    def clear(self) -> None:
+        """Drop all buckets and stats (fresh state, same config)."""
+        with self._lock:
+            self._buckets.clear()
+            self._stats.clear()

tool_throttle-0.1.0/tool_throttle.egg-info/PKG-INFO

@@ -0,0 +1,132 @@
+Metadata-Version: 2.4
+Name: tool-throttle
+Version: 0.1.0
+Summary: MCP tool-call throttling: token-bucket rate limits per tool name, a fuse against runaway agents
+Author: hao li
+License: MIT
+Project-URL: Homepage, https://github.com/hahahahahahahahah6/tool-throttle
+Keywords: mcp,ai-agents,rate-limit,throttling,token-bucket
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# tool-throttle
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue.svg)](pyproject.toml)
+[![No dependencies](https://img.shields.io/badge/dependencies-zero-brightgreen.svg)](pyproject.toml)
+[![PyPI](https://img.shields.io/pypi/v/tool-throttle.svg)](https://pypi.org/project/tool-throttle/)
+
+A token-bucket rate limiter for **MCP tool calls** — a fuse against runaway
+agents. When an agent gets stuck in a loop hammering `web_search` 400 times a
+minute, tool-throttle slows it down or stops it, per tool name, with per-tool
+quotas you configure in JSON.
+
+Zero dependencies. Standard library only. Thread-safe.
+
+```bash
+pip install tool-throttle
+```
+
+```python
+from tool_throttle import Throttler, ThrottlerConfig, Quota, REJECT
+
+th = Throttler(ThrottlerConfig(
+    default=Quota(calls=60, per_seconds=60),      # 60 calls/min for everything
+    per_tool={
+        "web_search":  Quota(calls=10, per_seconds=60),  # tight leash
+        "expensive_*": Quota(calls=2, per_seconds=60),   # glob patterns work
+    },
+    on_exceed=REJECT,   # or "queue": wait for a token, up to max_queue_wait
+))
+
+# wrap any callable -- your MCP tool dispatcher, a client method, anything:
+safe_search = th.wrap("web_search", mcp_client.call_tool)
+safe_search("web_search", {"query": "..."})   # -> ThrottledError past quota
+
+# or decide explicitly without consuming a token:
+d = th.decide("web_search")
+# Decision(tool='web_search', allowed=False, wait_seconds=42.1, ...)
+
+th.stats()  # per-tool allowed / rejected / queued / waited_seconds
+```
+
+Config round-trips through JSON, so quotas can live in a file:
+
+```python
+cfg = ThrottlerConfig(
+    default=Quota(calls=60, per_seconds=60),
+    per_tool={"web_*": Quota(calls=10, per_seconds=60, burst=3)},
+    on_exceed="queue",
+    max_queue_wait=30.0,
+)
+cfg.save("throttle.json")
+```
+
+## CLI
+
+```bash
+# dry-run: print a decision per {"tool": ...} JSON line on stdin
+echo '{"tool": "web_search"}' | tool-throttle check --config throttle.json
+
+# enforce: consume tokens (queues or rejects over-quota calls)
+echo '{"tool": "web_search"}' | tool-throttle acquire --config throttle.json
+
+# built-in demo of a looping agent being fused
+tool-throttle demo
+```
+
+## How it works
+
+Each distinct tool name gets its own `TokenBucket`, sized by the first
+matching quota in `per_tool` (glob patterns, insertion order) or `default`.
+Buckets refill steadily over time, so a `Quota(calls=10, per_seconds=60)`
+allows 10 calls per minute sustained, with a burst of up to `burst`
+(default: `calls`).
+
+On quota exhaustion:
+- `on_exceed="reject"` (default in the demo): raise `ThrottledError` with a
+  `retry_after` hint — fail fast.
+- `on_exceed="queue"`: block until a token frees up, bounded by
+  `max_queue_wait` seconds; if the wait would exceed the bound, raise
+  `ThrottledError(reason="queue_wait_exceeded")`.
+
+## Differentiation: why not just rate-limit at the API gateway?
+
+API gateways throttle **HTTP requests**. That is the wrong layer for agent
+safety:
+
+- One HTTP request can fan out into many tool calls; one tool call pattern
+  can span many HTTP endpoints. Gateway counters never see *tool-call
+  semantics* — they see bytes and paths.
+- Gateway limits are per API key / per IP. An agent looping on a *single
+  expensive tool* (say, a search tool that costs $0.02 per call) looks
+  perfectly normal at the HTTP layer while burning money at the tool layer.
+- Quotas here are expressed in the language of the agent: *tool names and
+  glob patterns* (`expensive_*`), with per-tool bursts and queue-or-reject
+  policies — the thing an agent developer actually wants to tune.
+
+tool-throttle is the **fuse at the MCP tool-call semantic layer**: it sits
+between the agent's tool-call intent and execution, trips before the loop
+does real damage, and reports per-tool stats so you can see which tool the
+agent was abusing. It complements gateway rate limiting; it does not
+replace it.
+
+## API reference
+
+| Symbol | Description |
+|---|---|
+| `Throttler(config)` | the interceptor; one per agent/process |
+| `ThrottlerConfig(default, per_tool, on_exceed, max_queue_wait)` | quotas + policy |
+| `Quota(calls, per_seconds=60, burst=None)` | rate quota for one pattern |
+| `th.acquire(tool)` | consume a token; queues or raises `ThrottledError` |
+| `th.decide(tool)` | dry-run check, does not consume |
+| `th.wrap(tool, fn)` / `th.call(tool, fn, *a, **k)` | throttle a callable |
+| `th.stats()` / `th.reset(tool?)` / `th.clear()` | observability and reset |
+| `ThrottledError` | `.tool`, `.retry_after`, `.reason` |
+| `TokenBucket(capacity, refill_per_second, clock=...)` | the primitive underneath |
+
+## License
+
+MIT — see [LICENSE](LICENSE).

tool_throttle-0.1.0/tool_throttle.egg-info/SOURCES.txt

@@ -0,0 +1,14 @@
+LICENSE
+README.md
+pyproject.toml
+tests/test_smoke.py
+tool_throttle/__init__.py
+tool_throttle/bucket.py
+tool_throttle/cli.py
+tool_throttle/config.py
+tool_throttle/throttler.py
+tool_throttle.egg-info/PKG-INFO
+tool_throttle.egg-info/SOURCES.txt
+tool_throttle.egg-info/dependency_links.txt
+tool_throttle.egg-info/entry_points.txt
+tool_throttle.egg-info/top_level.txt

tool_throttle-0.1.0/tool_throttle.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

tool_throttle-0.1.0/tool_throttle.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+tool-throttle = tool_throttle.cli:main

tool_throttle-0.1.0/tool_throttle.egg-info/top_level.txt

@@ -0,0 +1 @@
+tool_throttle