mcp-backpressure 0.1.2__tar.gz

mcp_backpressure-0.1.2/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 nulone
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mcp_backpressure-0.1.2/PKG-INFO

@@ -0,0 +1,220 @@
+Metadata-Version: 2.4
+Name: mcp-backpressure
+Version: 0.1.2
+Summary: Backpressure/concurrency control middleware for FastMCP MCP servers
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastmcp>=2.9.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Requires-Dist: ruff>=0.1; extra == "dev"
+Requires-Dist: mypy>=1.0; extra == "dev"
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
+Requires-Dist: ruff>=0.4; extra == "dev"
+Requires-Dist: mypy>=1.10; extra == "dev"
+Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Dynamic: license-file
+
+# mcp-backpressure
+
+Backpressure and concurrency control middleware for [FastMCP](https://github.com/jlowin/fastmcp) MCP servers.
+
+**Problem:** LLMs can generate hundreds of parallel tool calls, causing resource exhaustion, server crashes, and no structured feedback for clients to retry.
+
+**Solution:** Middleware that limits concurrent executions, queues excess requests with timeout, and returns structured JSON-RPC overload errors.
+
+## Quickstart
+
+```python
+from fastmcp import FastMCP
+from mcp_backpressure import BackpressureMiddleware
+
+mcp = FastMCP("MyServer")
+mcp.add_middleware(BackpressureMiddleware(
+    max_concurrent=5,      # Max parallel executions
+    queue_size=10,         # Bounded queue for waiting requests
+    queue_timeout=30.0,    # Queue wait timeout (seconds)
+))
+```
+
+## Installation
+
+```bash
+pip install mcp-backpressure
+```
+
+## Features
+
+- **Concurrency limiting**: Semaphore-based control of parallel executions
+- **Bounded queue**: Optional FIFO queue with configurable size
+- **Queue timeout**: Automatic timeout for queued requests with cleanup
+- **Structured errors**: JSON-RPC compliant overload errors with detailed metrics
+- **Metrics**: Real-time counters for active, queued, and rejected requests
+- **Callback hook**: Optional notification on each overload event
+- **Zero dependencies**: Only requires FastMCP and Python 3.10+
+
+## Usage
+
+### Basic Configuration
+
+```python
+from mcp_backpressure import BackpressureMiddleware
+
+mcp.add_middleware(BackpressureMiddleware(
+    max_concurrent=5,           # Required: max parallel tool executions
+    queue_size=10,              # Optional: bounded queue (0 = no queue)
+    queue_timeout=30.0,         # Optional: seconds to wait in queue
+    overload_error_code=-32001, # Optional: JSON-RPC error code
+    on_overload=callback,       # Optional: called on each overload
+))
+```
+
+### Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `max_concurrent` | `int` | **required** | Maximum number of concurrent tool executions. Must be >= 1. |
+| `queue_size` | `int` | `0` | Maximum queue size for waiting requests. Set to 0 to reject immediately when limit reached. |
+| `queue_timeout` | `float` | `30.0` | Maximum time (seconds) a request can wait in queue before timing out. Must be > 0. |
+| `overload_error_code` | `int` | `-32001` | JSON-RPC error code returned when server is overloaded. |
+| `on_overload` | `Callable` | `None` | Optional callback `(error: OverloadError) -> None` invoked on each overload. |
+
+### Error Handling
+
+When the server is overloaded, requests are rejected with a structured JSON-RPC error:
+
+```json
+{
+  "code": -32001,
+  "message": "SERVER_OVERLOADED",
+  "data": {
+    "reason": "queue_full",
+    "active": 5,
+    "queued": 10,
+    "max_concurrent": 5,
+    "queue_size": 10,
+    "queue_timeout_ms": 30000,
+    "retry_after_ms": 1000
+  }
+}
+```
+
+#### Overload Reasons
+
+| Reason | Description |
+|--------|-------------|
+| `concurrency_limit` | All execution slots full and no queue configured (queue_size=0) |
+| `queue_full` | All execution slots and queue slots are full |
+| `queue_timeout` | Request waited in queue longer than `queue_timeout` |
+
+### Metrics
+
+Get real-time metrics from the middleware:
+
+```python
+metrics = middleware.get_metrics()  # Synchronous
+
+print(f"Active: {metrics.active}")
+print(f"Queued: {metrics.queued}")
+print(f"Total rejected: {metrics.total_rejected}")
+print(f"Rejected (concurrency): {metrics.rejected_concurrency_limit}")
+print(f"Rejected (queue full): {metrics.rejected_queue_full}")
+print(f"Rejected (timeout): {metrics.rejected_queue_timeout}")
+```
+
+For async contexts, use `await middleware.get_metrics_async()`.
+
+### Callback Hook
+
+Register a callback to be notified of each overload event:
+
+```python
+def on_overload(error: OverloadError):
+    print(f"OVERLOAD: {error.reason} (active={error.active})")
+    # Log to monitoring system, update metrics, etc.
+
+middleware = BackpressureMiddleware(
+    max_concurrent=5,
+    queue_size=10,
+    on_overload=on_overload,
+)
+```
+
+## Examples
+
+### Simple Server
+
+See [examples/simple_server.py](examples/simple_server.py) for a minimal FastMCP server with backpressure.
+
+### Load Simulation
+
+Run [examples/load_simulation.py](examples/load_simulation.py) to see backpressure behavior under heavy concurrent load:
+
+```bash
+python examples/load_simulation.py
+```
+
+This simulates 30 concurrent requests against a server limited to 5 concurrent executions with a queue of 10, demonstrating how the middleware handles overload.
+
+## How It Works
+
+The middleware provides two-level limiting:
+
+1. **Semaphore** (max_concurrent): Controls active executions
+2. **Bounded queue** (queue_size): Holds waiting requests with timeout
+
+**Request flow:**
+- If execution slot available → execute immediately
+- If execution slots full and queue not full → wait in queue with timeout
+- If queue full → reject with `queue_full`
+- If timeout in queue → reject with `queue_timeout`
+
+**Invariants** (guaranteed under all conditions):
+- `active <= max_concurrent` ALWAYS
+- `queued <= queue_size` ALWAYS
+- Cancellation correctly frees slots and decrements counters
+- Queue timeout removes item from queue
+
+## Development
+
+### Running Tests
+
+```bash
+python -m pytest tests/ -v
+```
+
+### Linting
+
+```bash
+ruff check src/ tests/
+```
+
+## Design Rationale
+
+This library emerged from [python-sdk #1698](https://github.com/modelcontextprotocol/python-sdk/issues/1698) (closed as "not planned"). Key design decisions:
+
+- **Global limits only** (v0.1): Per-client and per-tool limits deferred to v0.2+
+- **Simple counters**: No Prometheus/OTEL dependencies by default
+- **JSON-RPC errors**: Follows MCP protocol conventions
+- **Monotonic time**: Queue timeouts use `time.monotonic()` for reliability
+
+## License
+
+MIT
+
+## Contributing
+
+Contributions welcome! Please open an issue before submitting PRs.
+
+## Changelog
+
+See [CHANGELOG.md](https://github.com/nulone/mcp-backpressure/blob/main/CHANGELOG.md)

mcp_backpressure-0.1.2/README.md

@@ -0,0 +1,194 @@
+# mcp-backpressure
+
+Backpressure and concurrency control middleware for [FastMCP](https://github.com/jlowin/fastmcp) MCP servers.
+
+**Problem:** LLMs can generate hundreds of parallel tool calls, causing resource exhaustion, server crashes, and no structured feedback for clients to retry.
+
+**Solution:** Middleware that limits concurrent executions, queues excess requests with timeout, and returns structured JSON-RPC overload errors.
+
+## Quickstart
+
+```python
+from fastmcp import FastMCP
+from mcp_backpressure import BackpressureMiddleware
+
+mcp = FastMCP("MyServer")
+mcp.add_middleware(BackpressureMiddleware(
+    max_concurrent=5,      # Max parallel executions
+    queue_size=10,         # Bounded queue for waiting requests
+    queue_timeout=30.0,    # Queue wait timeout (seconds)
+))
+```
+
+## Installation
+
+```bash
+pip install mcp-backpressure
+```
+
+## Features
+
+- **Concurrency limiting**: Semaphore-based control of parallel executions
+- **Bounded queue**: Optional FIFO queue with configurable size
+- **Queue timeout**: Automatic timeout for queued requests with cleanup
+- **Structured errors**: JSON-RPC compliant overload errors with detailed metrics
+- **Metrics**: Real-time counters for active, queued, and rejected requests
+- **Callback hook**: Optional notification on each overload event
+- **Zero dependencies**: Only requires FastMCP and Python 3.10+
+
+## Usage
+
+### Basic Configuration
+
+```python
+from mcp_backpressure import BackpressureMiddleware
+
+mcp.add_middleware(BackpressureMiddleware(
+    max_concurrent=5,           # Required: max parallel tool executions
+    queue_size=10,              # Optional: bounded queue (0 = no queue)
+    queue_timeout=30.0,         # Optional: seconds to wait in queue
+    overload_error_code=-32001, # Optional: JSON-RPC error code
+    on_overload=callback,       # Optional: called on each overload
+))
+```
+
+### Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `max_concurrent` | `int` | **required** | Maximum number of concurrent tool executions. Must be >= 1. |
+| `queue_size` | `int` | `0` | Maximum queue size for waiting requests. Set to 0 to reject immediately when limit reached. |
+| `queue_timeout` | `float` | `30.0` | Maximum time (seconds) a request can wait in queue before timing out. Must be > 0. |
+| `overload_error_code` | `int` | `-32001` | JSON-RPC error code returned when server is overloaded. |
+| `on_overload` | `Callable` | `None` | Optional callback `(error: OverloadError) -> None` invoked on each overload. |
+
+### Error Handling
+
+When the server is overloaded, requests are rejected with a structured JSON-RPC error:
+
+```json
+{
+  "code": -32001,
+  "message": "SERVER_OVERLOADED",
+  "data": {
+    "reason": "queue_full",
+    "active": 5,
+    "queued": 10,
+    "max_concurrent": 5,
+    "queue_size": 10,
+    "queue_timeout_ms": 30000,
+    "retry_after_ms": 1000
+  }
+}
+```
+
+#### Overload Reasons
+
+| Reason | Description |
+|--------|-------------|
+| `concurrency_limit` | All execution slots full and no queue configured (queue_size=0) |
+| `queue_full` | All execution slots and queue slots are full |
+| `queue_timeout` | Request waited in queue longer than `queue_timeout` |
+
+### Metrics
+
+Get real-time metrics from the middleware:
+
+```python
+metrics = middleware.get_metrics()  # Synchronous
+
+print(f"Active: {metrics.active}")
+print(f"Queued: {metrics.queued}")
+print(f"Total rejected: {metrics.total_rejected}")
+print(f"Rejected (concurrency): {metrics.rejected_concurrency_limit}")
+print(f"Rejected (queue full): {metrics.rejected_queue_full}")
+print(f"Rejected (timeout): {metrics.rejected_queue_timeout}")
+```
+
+For async contexts, use `await middleware.get_metrics_async()`.
+
+### Callback Hook
+
+Register a callback to be notified of each overload event:
+
+```python
+def on_overload(error: OverloadError):
+    print(f"OVERLOAD: {error.reason} (active={error.active})")
+    # Log to monitoring system, update metrics, etc.
+
+middleware = BackpressureMiddleware(
+    max_concurrent=5,
+    queue_size=10,
+    on_overload=on_overload,
+)
+```
+
+## Examples
+
+### Simple Server
+
+See [examples/simple_server.py](examples/simple_server.py) for a minimal FastMCP server with backpressure.
+
+### Load Simulation
+
+Run [examples/load_simulation.py](examples/load_simulation.py) to see backpressure behavior under heavy concurrent load:
+
+```bash
+python examples/load_simulation.py
+```
+
+This simulates 30 concurrent requests against a server limited to 5 concurrent executions with a queue of 10, demonstrating how the middleware handles overload.
+
+## How It Works
+
+The middleware provides two-level limiting:
+
+1. **Semaphore** (max_concurrent): Controls active executions
+2. **Bounded queue** (queue_size): Holds waiting requests with timeout
+
+**Request flow:**
+- If execution slot available → execute immediately
+- If execution slots full and queue not full → wait in queue with timeout
+- If queue full → reject with `queue_full`
+- If timeout in queue → reject with `queue_timeout`
+
+**Invariants** (guaranteed under all conditions):
+- `active <= max_concurrent` ALWAYS
+- `queued <= queue_size` ALWAYS
+- Cancellation correctly frees slots and decrements counters
+- Queue timeout removes item from queue
+
+## Development
+
+### Running Tests
+
+```bash
+python -m pytest tests/ -v
+```
+
+### Linting
+
+```bash
+ruff check src/ tests/
+```
+
+## Design Rationale
+
+This library emerged from [python-sdk #1698](https://github.com/modelcontextprotocol/python-sdk/issues/1698) (closed as "not planned"). Key design decisions:
+
+- **Global limits only** (v0.1): Per-client and per-tool limits deferred to v0.2+
+- **Simple counters**: No Prometheus/OTEL dependencies by default
+- **JSON-RPC errors**: Follows MCP protocol conventions
+- **Monotonic time**: Queue timeouts use `time.monotonic()` for reliability
+
+## License
+
+MIT
+
+## Contributing
+
+Contributions welcome! Please open an issue before submitting PRs.
+
+## Changelog
+
+See [CHANGELOG.md](https://github.com/nulone/mcp-backpressure/blob/main/CHANGELOG.md)

mcp_backpressure-0.1.2/pyproject.toml

@@ -0,0 +1,83 @@
+[build-system]
+requires = [
+    "setuptools>=68.0",
+    "wheel",
+]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "mcp-backpressure"
+version = "0.1.2"
+description = "Backpressure/concurrency control middleware for FastMCP MCP servers"
+readme = "README.md"
+requires-python = ">=3.10"
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+]
+dependencies = [
+    "fastmcp>=2.9.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0",
+    "pytest-cov>=4.0",
+    "ruff>=0.1",
+    "mypy>=1.0",
+    "pytest>=8.0",
+    "pytest-asyncio>=0.23",
+    "ruff>=0.4",
+    "mypy>=1.10",
+    "pytest-cov>=5.0",
+]
+
+[tool.setuptools.packages.find]
+where = [
+    "src",
+]
+
+[tool.pytest.ini_options]
+testpaths = [
+    "tests",
+]
+python_files = [
+    "test_*.py",
+]
+python_classes = [
+    "Test*",
+]
+python_functions = [
+    "test_*",
+]
+addopts = "-v --strict-markers --tb=short"
+asyncio_mode = "auto"
+asyncio_default_fixture_loop_scope = "function"
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+
+[tool.ruff.lint]
+select = [
+    "E",
+    "W",
+    "F",
+    "I",
+    "B",
+    "C4",
+    "UP",
+]
+ignore = [
+    "E501",
+]
+
+[tool.mypy]
+python_version = "3.10"
+strict = true
+warn_return_any = true
+warn_unused_configs = true

mcp_backpressure-0.1.2/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

mcp_backpressure-0.1.2/src/mcp_backpressure/__init__.py

@@ -0,0 +1,13 @@
+"""Backpressure/concurrency control middleware for FastMCP MCP servers"""
+
+__version__ = "0.1.0"
+
+from .errors import OverloadError
+from .metrics import BackpressureMetrics
+from .middleware import BackpressureMiddleware
+
+__all__ = [
+    "BackpressureMiddleware",
+    "BackpressureMetrics",
+    "OverloadError",
+]

mcp_backpressure-0.1.2/src/mcp_backpressure/errors.py

@@ -0,0 +1,75 @@
+"""Error classes for mcp-backpressure middleware"""
+
+from typing import Any
+
+
+class OverloadError(Exception):
+    """
+    Raised when server is overloaded and cannot accept more requests.
+
+    Follows JSON-RPC error format for MCP protocol.
+    """
+
+    def __init__(
+        self,
+        reason: str,
+        active: int,
+        max_concurrent: int,
+        code: int = -32001,
+        message: str = "SERVER_OVERLOADED",
+        queued: int = 0,
+        queue_size: int = 0,
+        queue_timeout_ms: int = 0,
+        retry_after_ms: int = 1000,
+    ):
+        """
+        Create an OverloadError.
+
+        Args:
+            reason: Overload reason ('concurrency_limit', 'queue_full', 'queue_timeout')
+            active: Number of currently active requests
+            max_concurrent: Maximum allowed concurrent requests
+            code: JSON-RPC error code (default: -32001)
+            message: Error message (default: 'SERVER_OVERLOADED')
+            queued: Number of requests in queue (default: 0)
+            queue_size: Maximum queue size (default: 0)
+            queue_timeout_ms: Queue timeout in milliseconds (default: 0)
+            retry_after_ms: Suggested retry delay in milliseconds (default: 1000)
+        """
+        self.code = code
+        self.message = message
+        self.reason = reason
+        self.active = active
+        self.max_concurrent = max_concurrent
+        self.queued = queued
+        self.queue_size = queue_size
+        self.queue_timeout_ms = queue_timeout_ms
+        self.retry_after_ms = retry_after_ms
+
+        super().__init__(f"{message}: {reason}")
+
+    @property
+    def data(self) -> dict[str, Any]:
+        """Get error data payload."""
+        return {
+            "reason": self.reason,
+            "active": self.active,
+            "queued": self.queued,
+            "max_concurrent": self.max_concurrent,
+            "queue_size": self.queue_size,
+            "queue_timeout_ms": self.queue_timeout_ms,
+            "retry_after_ms": self.retry_after_ms,
+        }
+
+    def to_json_rpc(self) -> dict[str, Any]:
+        """
+        Convert to JSON-RPC error object.
+
+        Returns:
+            dict with 'code', 'message', and 'data' keys
+        """
+        return {
+            "code": self.code,
+            "message": self.message,
+            "data": self.data,
+        }