throttlekit 0.1.0__tar.gz

throttlekit-0.1.0/PKG-INFO

@@ -0,0 +1,205 @@
+Metadata-Version: 2.4
+Name: throttlekit
+Version: 0.1.0
+Summary: Fast asyncio-compatible token bucket rate limiter
+Author: Roudrasekhar Majumder
+Author-email: Roudrasekhar Majumder <roudra25@gmail.com>
+License-Expression: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries
+Requires-Dist: typing-extensions>=4.0.0 ; python_full_version < '3.10'
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+
+# 🔄 throttlekit
+
+A lightweight, asyncio-based rate limiting library for Python that provides flexible and efficient rate limiting solutions.
+
+## 📋 Overview
+
+**throttlekit** offers two proven rate limiting algorithms:
+
+- **⚡ TokenBucketRateLimiter**: Allows controlled bursts of activity
+- **💧 LeakyBucketRateLimiter**: Enforces a strict, steady rate
+
+Perfect for API throttling, web scrapers, background jobs, and queue management.
+
+## 🚀 Features
+
+- ✅ **Two proven algorithms**: Token Bucket (burst-tolerant) and Leaky Bucket (evenly-paced)
+- ✅ **Multiple usage patterns**: `@decorator`, `async with`, and manual `.acquire()`
+- ✅ **Concurrency control**: Optional `concurrency_limit` parameter
+- ✅ **High performance**: Low-overhead design optimized for async workloads
+- ✅ **asyncio integration**: Works seamlessly with `asyncio.gather()` and `TaskGroup`
+
+## ⚙️ Installation
+
+### Using uv (recommended)
+
+```bash
+uv add throttlekit
+```
+
+### Using pip
+
+```bash
+pip install throttlekit
+```
+
+## ✨ Quick Start
+
+```python
+import asyncio
+from throttlekit import TokenBucketRateLimiter
+
+# Create a rate limiter (5 tokens, refill every second)
+limiter = TokenBucketRateLimiter(max_tokens=5, refill_interval=1.0)
+
+@limiter.limit
+async def call_api(i):
+    await asyncio.sleep(0.2)
+    return f"Request {i} completed"
+
+async def main():
+    await limiter.start()
+    
+    # Process 10 requests with rate limiting
+    results = await asyncio.gather(*(call_api(i) for i in range(10)))
+    print(results)
+
+# Run the example
+asyncio.run(main())
+```
+
+## 🧠 Which Limiter Should I Use?
+
+| Use Case | Recommended Limiter | Why? |
+|----------|-------------------|------|
+| Allow short bursts (e.g., 5 calls at once) |  **Token Bucket** | Accumulates tokens for burst capacity |
+| Require steady pacing (e.g., 1 call/sec max) |  **Leaky Bucket** | Maintains consistent rate |
+| Queue smoothing, task draining |  **Leaky Bucket** | FIFO processing at fixed rate |
+| Per-user or per-key API quotas |  **Token Bucket** | Flexible burst handling |
+
+## Rate Limiters
+
+### TokenBucketRateLimiter
+
+Allows bursts up to `max_tokens`, then refills at a steady rate.
+
+```python
+from throttlekit import TokenBucketRateLimiter
+
+limiter = TokenBucketRateLimiter(
+    max_tokens=10,           # Maximum burst size
+    refill_interval=1.0,     # Refill every second
+    concurrency_limit=5      # Optional: limit concurrent operations
+)
+```
+
+**Supports:**
+
+- `@limiter.limit` decorator
+- `async with limiter` context manager  
+- `await limiter.acquire()` manual usage
+
+### 💧 LeakyBucketRateLimiter
+
+Processes requests at a fixed rate, queuing excess requests.
+
+```python
+from throttlekit import LeakyBucketRateLimiter
+
+limiter = LeakyBucketRateLimiter(
+    rate=2.0,              # 2 requests per second
+    max_queue_size=100     # Maximum queued requests
+)
+```
+
+**Behavior:**
+
+- Drains 1 request every `1/rate` seconds in FIFO order
+- Queued requests are processed at a fixed rate
+- Bursts are automatically queued (up to `max_queue_size`)
+
+## 📘 Usage Examples
+
+### 1️⃣ Decorator Pattern
+
+```python
+@limiter.limit
+async def fetch_data(url):
+    async with aiohttp.ClientSession() as session:
+        async with session.get(url) as response:
+            return await response.json()
+```
+
+### 2️⃣ Context Manager
+
+```python
+async with limiter:
+    result = await expensive_operation()
+    return result
+```
+
+### 3️⃣ Manual Control
+
+```python
+await limiter.acquire()
+try:
+    result = await do_work()
+finally:
+    limiter.release_token()      # For TokenBucket
+    limiter.release_semaphore()  # If using concurrency_limit
+```
+
+## 🧪 Testing
+
+Install test dependencies:
+
+```bash
+uv pip install pytest pytest-asyncio
+```
+
+Run tests with coverage:
+
+```bash
+pytest --cov=src/throttlekit --cov-report=term-missing
+```
+
+## 📁 Project Structure
+
+```tree
+throttlekit/
+├── src/
+│   └── throttlekit/
+│       ├── __init__.py
+│       ├── limiter.py           # TokenBucketRateLimiter
+│       └── leaky_limiter.py     # LeakyBucketRateLimiter
+├── tests/
+│   ├── test_token_bucket_limiter.py
+│   └── test_leaky_bucket_limiter.py
+├── pyproject.toml
+├── README.md
+└── LICENSE
+```
+
+## 📜 License
+
+MIT License © [Roudrasekhar Majumder](https://github.com/rowds)
+
+## 🙋 Contributing
+
+We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed setup instructions and guidelines.
+
+---
+
+**⭐ Star this repo if you find it useful!**
+
+[Report Bug](https://github.com/rowds/throttlekit/issues) • [Request Feature](https://github.com/rowds/throttlekit/issues) • [Documentation](https://github.com/rowds/throttlekit)

throttlekit-0.1.0/README.md

@@ -0,0 +1,185 @@
+# 🔄 throttlekit
+
+A lightweight, asyncio-based rate limiting library for Python that provides flexible and efficient rate limiting solutions.
+
+## 📋 Overview
+
+**throttlekit** offers two proven rate limiting algorithms:
+
+- **⚡ TokenBucketRateLimiter**: Allows controlled bursts of activity
+- **💧 LeakyBucketRateLimiter**: Enforces a strict, steady rate
+
+Perfect for API throttling, web scrapers, background jobs, and queue management.
+
+## 🚀 Features
+
+- ✅ **Two proven algorithms**: Token Bucket (burst-tolerant) and Leaky Bucket (evenly-paced)
+- ✅ **Multiple usage patterns**: `@decorator`, `async with`, and manual `.acquire()`
+- ✅ **Concurrency control**: Optional `concurrency_limit` parameter
+- ✅ **High performance**: Low-overhead design optimized for async workloads
+- ✅ **asyncio integration**: Works seamlessly with `asyncio.gather()` and `TaskGroup`
+
+## ⚙️ Installation
+
+### Using uv (recommended)
+
+```bash
+uv add throttlekit
+```
+
+### Using pip
+
+```bash
+pip install throttlekit
+```
+
+## ✨ Quick Start
+
+```python
+import asyncio
+from throttlekit import TokenBucketRateLimiter
+
+# Create a rate limiter (5 tokens, refill every second)
+limiter = TokenBucketRateLimiter(max_tokens=5, refill_interval=1.0)
+
+@limiter.limit
+async def call_api(i):
+    await asyncio.sleep(0.2)
+    return f"Request {i} completed"
+
+async def main():
+    await limiter.start()
+    
+    # Process 10 requests with rate limiting
+    results = await asyncio.gather(*(call_api(i) for i in range(10)))
+    print(results)
+
+# Run the example
+asyncio.run(main())
+```
+
+## 🧠 Which Limiter Should I Use?
+
+| Use Case | Recommended Limiter | Why? |
+|----------|-------------------|------|
+| Allow short bursts (e.g., 5 calls at once) |  **Token Bucket** | Accumulates tokens for burst capacity |
+| Require steady pacing (e.g., 1 call/sec max) |  **Leaky Bucket** | Maintains consistent rate |
+| Queue smoothing, task draining |  **Leaky Bucket** | FIFO processing at fixed rate |
+| Per-user or per-key API quotas |  **Token Bucket** | Flexible burst handling |
+
+## Rate Limiters
+
+### TokenBucketRateLimiter
+
+Allows bursts up to `max_tokens`, then refills at a steady rate.
+
+```python
+from throttlekit import TokenBucketRateLimiter
+
+limiter = TokenBucketRateLimiter(
+    max_tokens=10,           # Maximum burst size
+    refill_interval=1.0,     # Refill every second
+    concurrency_limit=5      # Optional: limit concurrent operations
+)
+```
+
+**Supports:**
+
+- `@limiter.limit` decorator
+- `async with limiter` context manager  
+- `await limiter.acquire()` manual usage
+
+### 💧 LeakyBucketRateLimiter
+
+Processes requests at a fixed rate, queuing excess requests.
+
+```python
+from throttlekit import LeakyBucketRateLimiter
+
+limiter = LeakyBucketRateLimiter(
+    rate=2.0,              # 2 requests per second
+    max_queue_size=100     # Maximum queued requests
+)
+```
+
+**Behavior:**
+
+- Drains 1 request every `1/rate` seconds in FIFO order
+- Queued requests are processed at a fixed rate
+- Bursts are automatically queued (up to `max_queue_size`)
+
+## 📘 Usage Examples
+
+### 1️⃣ Decorator Pattern
+
+```python
+@limiter.limit
+async def fetch_data(url):
+    async with aiohttp.ClientSession() as session:
+        async with session.get(url) as response:
+            return await response.json()
+```
+
+### 2️⃣ Context Manager
+
+```python
+async with limiter:
+    result = await expensive_operation()
+    return result
+```
+
+### 3️⃣ Manual Control
+
+```python
+await limiter.acquire()
+try:
+    result = await do_work()
+finally:
+    limiter.release_token()      # For TokenBucket
+    limiter.release_semaphore()  # If using concurrency_limit
+```
+
+## 🧪 Testing
+
+Install test dependencies:
+
+```bash
+uv pip install pytest pytest-asyncio
+```
+
+Run tests with coverage:
+
+```bash
+pytest --cov=src/throttlekit --cov-report=term-missing
+```
+
+## 📁 Project Structure
+
+```tree
+throttlekit/
+├── src/
+│   └── throttlekit/
+│       ├── __init__.py
+│       ├── limiter.py           # TokenBucketRateLimiter
+│       └── leaky_limiter.py     # LeakyBucketRateLimiter
+├── tests/
+│   ├── test_token_bucket_limiter.py
+│   └── test_leaky_bucket_limiter.py
+├── pyproject.toml
+├── README.md
+└── LICENSE
+```
+
+## 📜 License
+
+MIT License © [Roudrasekhar Majumder](https://github.com/rowds)
+
+## 🙋 Contributing
+
+We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed setup instructions and guidelines.
+
+---
+
+**⭐ Star this repo if you find it useful!**
+
+[Report Bug](https://github.com/rowds/throttlekit/issues) • [Request Feature](https://github.com/rowds/throttlekit/issues) • [Documentation](https://github.com/rowds/throttlekit)

throttlekit-0.1.0/pyproject.toml

@@ -0,0 +1,42 @@
+[project]
+name = "throttlekit"
+version = "0.1.0"
+description = "Fast asyncio-compatible token bucket rate limiter"
+readme = "README.md"
+requires-python = ">=3.9"
+authors = [{ name = "Roudrasekhar Majumder", email = "roudra25@gmail.com" }]
+license = "MIT"
+dependencies = [
+    "typing-extensions>=4.0.0; python_version<'3.10'"
+]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Intended Audience :: Developers",
+    "Topic :: Software Development :: Libraries"
+]
+
+
+[build-system]
+requires = ["uv_build>=0.8.3"]
+build-backend = "uv_build"
+
+[tool.uv.build-backend]
+name = "throttlekit"
+path = "src"
+packages = ["throttlekit"]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+
+[tool.uv]
+dev-dependencies = [
+    "pytest>=8.4.1",
+    "pytest-asyncio>=1.1.0",
+    "pytest-cov>=6.2.1"
+]

throttlekit-0.1.0/src/throttlekit/__init__.py

@@ -0,0 +1,2 @@
+from .limiter import TokenBucketRateLimiter
+from .leaky_limiter import LeakyBucketRateLimiter

throttlekit-0.1.0/src/throttlekit/leaky_limiter.py

@@ -0,0 +1,72 @@
+import asyncio
+import logging
+from functools import wraps
+from typing import Callable, Awaitable, Optional, TypeVar, Union
+
+try:
+    from typing import ParamSpec
+except ImportError:
+    from typing_extensions import ParamSpec  # type: ignore
+
+T = TypeVar("T")
+P = ParamSpec("P")
+
+class LeakyBucketRateLimiter:
+    def __init__(
+        self,
+        rate: float = 1.0,  # requests per second
+        max_queue_size: int = 100,
+        logger: Optional[logging.Logger] = None
+    ) -> None:
+        if rate <= 0:
+            raise ValueError("rate must be > 0")
+        self.leak_interval = 1.0 / rate  # convert to delay between each request
+        self.queue: asyncio.Queue[asyncio.Future[None]] = asyncio.Queue(maxsize=max_queue_size)
+        self._started = False
+        self._logger = logger or logging.getLogger(__name__)
+        self._drain_task: Optional[asyncio.Task] = None
+
+    async def start(self) -> None:
+        if self._started:
+            return
+        self._started = True
+        self._drain_task = asyncio.create_task(self._drain_loop())
+        self._logger.debug("LeakyBucket started with leak_interval=%s", self.leak_interval)
+
+    async def _drain_loop(self) -> None:
+        while True:
+            fut = await self.queue.get()
+            if not fut.done():
+                fut.set_result(None)
+            await asyncio.sleep(self.leak_interval)
+
+    async def acquire(self) -> None:
+        if not self._started:
+            raise RuntimeError("LeakyBucket not started. Call await limiter.start() before use.")
+        fut: asyncio.Future[None] = asyncio.get_event_loop().create_future()
+        await self.queue.put(fut)
+        await fut
+
+    async def __aenter__(self) -> "LeakyBucketRateLimiter":
+        await self.acquire()
+        return self
+
+    async def __aexit__(self, exc_type, exc_val, exc_tb) -> Optional[bool]:
+        return None
+
+    def limit(
+        self,
+        fn: Optional[Callable[..., Awaitable[T]]] = None
+    ) -> Union[
+        Callable[[Callable[..., Awaitable[T]]], Callable[..., Awaitable[T]]],
+        Callable[..., Awaitable[T]]
+    ]:
+
+        def decorator(func: Callable[..., Awaitable[T]]) -> Callable[..., Awaitable[T]]:
+            @wraps(func)
+            async def wrapper(*args, **kwargs) -> T:
+                await self.acquire()
+                return await func(*args, **kwargs)
+            return wrapper
+
+        return decorator if fn is None else decorator(fn)

throttlekit-0.1.0/src/throttlekit/limiter.py

@@ -0,0 +1,104 @@
+import asyncio
+from functools import wraps
+from typing import Callable, Awaitable, TypeVar, Optional, Union
+import logging
+
+try:
+    from typing import ParamSpec
+except ImportError:
+    from typing_extensions import ParamSpec  # type: ignore
+
+T = TypeVar("T")
+P = ParamSpec("P")
+
+class TokenBucketRateLimiter:
+    def __init__(
+        self,
+        max_tokens: int,
+        refill_interval: float = 1.0,
+        concurrency_limit: Optional[int] = None,
+        logger: Optional[logging.Logger] = None
+    ) -> None:
+        self.max_tokens: int = max_tokens
+        self.refill_interval: float = refill_interval
+        self.bucket: asyncio.Queue[int] = asyncio.Queue(maxsize=max_tokens)
+        self.semaphore: Optional[asyncio.Semaphore] = asyncio.Semaphore(concurrency_limit) if concurrency_limit else None
+        self.started: bool = False
+        self.logger: logging.Logger = logger or logging.getLogger(__name__)
+
+    async def start(self) -> None:
+        if self.started:
+            return
+        self.started = True
+        for _ in range(self.max_tokens):
+            self.bucket.put_nowait(1)
+        asyncio.create_task(self._refill())
+        self.logger.debug("Rate limiter started with %d tokens", self.max_tokens)
+
+    async def _refill(self) -> None:
+        while True:
+            await asyncio.sleep(self.refill_interval)
+            for _ in range(self.max_tokens - self.bucket.qsize()):
+                try:
+                    self.bucket.put_nowait(1)
+                except asyncio.QueueFull:
+                    break
+
+    async def acquire(self) -> None:
+        if not self.started:
+            raise RuntimeError("RateLimiter not started. Call await limiter.start() before use.")
+        await self.bucket.get()
+        if self.semaphore:
+            await self.semaphore.acquire()
+
+    def release_token(self) -> None:
+        """Refund a token manually (not normally used)."""
+        try:
+            self.bucket.put_nowait(1)
+        except asyncio.QueueFull:
+            pass
+
+    def release_semaphore(self) -> None:
+        """Manually release the semaphore if acquired outside context."""
+        if self.semaphore:
+            self.semaphore.release()
+
+    async def __aenter__(self) -> "TokenBucketRateLimiter":
+        await self.acquire()
+        return self
+
+    async def __aexit__(self, exc_type, exc_val, exc_tb) -> Optional[bool]:
+        self.release_semaphore()
+        return None
+    
+    def _release(self) -> None:
+        """Internal helper for releasing either semaphore or bucket."""
+        self.release_semaphore()
+        self.release_token()
+
+    def limit(
+        self,
+        fn: Optional[Callable[..., Awaitable[T]]] = None,
+        *,
+        tokens: int = 1
+    ) -> Union[
+        Callable[[Callable[..., Awaitable[T]]], Callable[..., Awaitable[T]]],
+        Callable[..., Awaitable[T]]
+    ]:
+
+        def decorator(func: Callable[..., Awaitable[T]]) -> Callable[..., Awaitable[T]]:
+            @wraps(func)
+            async def wrapper(*args, **kwargs) -> T:
+                for _ in range(tokens):
+                    await self.acquire()
+                try:
+                    return await func(*args, **kwargs)
+                finally:
+                    for _ in range(tokens):
+                        self._release()
+            return wrapper
+
+        if fn is None:
+            return decorator
+        else:
+            return decorator(fn)