httpx-limiter 0.2.0__tar.gz → 0.3.0__tar.gz

{httpx_limiter-0.2.0 → httpx_limiter-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: httpx-limiter
-Version: 0.2.0
+Version: 0.3.0
 Summary: A lightweight package that provides rate-limited httpx transports.
 Project-URL: Homepage, https://github.com/Midnighter/httpx-limiter
 Project-URL: Documentation, https://httpx-limiter.readthedocs.io
@@ -49,12 +49,16 @@ Install it, for example, with
 pip install httpx-limiter
 ```
 
-## Usage
+## Tutorial
 
-For situations when you need to make a large number of asynchronous request with
-a controlled number of requests per unit time, you can apply rate limiting to an
-HTTPX client using the provided transport. If you want to be able to make ten
-requests per second, for example, use the following:
+You can limit the number of requests made by an HTTPX client using the transports
+provided in this package. That is useful in situations when you need to make a large
+number of asynchronous requests against endpoints that implement a rate limit,
+
+### Single Rate Limit
+
+The simplest use case is to apply a single rate limit to all requests. If you want to be
+able to make twenty requests per second, for example, use the following code:
 
 ```python
 import httpx
@@ -62,11 +66,77 @@ from httpx_limiter import AsyncRateLimitedTransport, Rate
 
 async def main():
     async with httpx.AsyncClient(
-        transport=AsyncRateLimitedTransport.create(rate=Rate.create(magnitude=10)),
+        transport=AsyncRateLimitedTransport.create(rate=Rate.create(magnitude=20)),
     ) as client:
         response = await client.get("https://httpbin.org")
 ```
 
+> [!IMPORTANT]
+> Due to limitations in the design of the underlying leaky bucket implementation,
+> which is used to implement the rate limiting, the magnitude of the rate is also the
+> maximum capacity of the bucket. That means, if you set a rate that is larger than one,
+> a burst of requests equal to that capacity will be allowed. If you do not want to
+> allow any bursts, set the magnitude to one, but the duration to the inverse of your
+> desired rate. If you want to allow twenty requests per second, for example, set the
+> magnitude to 1 and the duration to 0.05 seconds.
+
+### Multiple Rate Limits
+
+For more advanced use cases, you can apply different rate limits based on a concrete
+implementation of the `AbstractRateLimiterRepository`. There are two relevant methods
+that both get passed the current request. One method needs to identify which rate limit
+to apply, and the other method sets the rate limit itself. See the following example:
+
+```python
+import httpx
+from httpx_limiter import AbstractRateLimiterRepository, AsyncMultiRateLimitedTransport, Rate
+
+class DomainBasedRateLimiterRepository(AbstractRateLimiterRepository):
+    """Apply different rate limits based on the domain being requested."""
+    
+    def get_identifier(self, request: httpx.Request) -> str:
+        """Return the domain as the identifier for rate limiting."""
+        return request.url.host
+    
+    def get_rate(self, request: httpx.Request) -> Rate:
+        """Apply the same, but independent rate limit to each domain."""
+        return Rate.create(magnitude=25)
+
+client = httpx.AsyncClient(
+    transport=AsyncMultiRateLimitedTransport.create(
+        repository=DomainBasedRateLimiterRepository(),
+    ),
+)
+```
+
+> [!NOTE]
+> You are free to ignore the request argument and use global information like the time
+> of day to determine the rate limit.
+
+```python
+from datetime import datetime, timezone
+
+import httpx
+from httpx_limiter import AbstractRateLimiterRepository, AsyncMultiRateLimitedTransport, Rate
+
+class DayNightRateLimiterRepository(AbstractRateLimiterRepository):
+    """Apply different rate limits based on the time of day."""
+
+    def get_identifier(self, _: httpx.Request) -> str:
+        """Identify whether it is currently day or night."""
+        if 6 <= datetime.now(tz=timezone.utc).hour < 18:
+            return "day"
+        
+        return "night"
+
+    def get_rate(self, _: httpx.Request) -> Rate:
+        """Apply different rate limits during the day or night."""
+        if self.get_identifier(_) == "day":
+            return Rate.create(magnitude=10)
+        
+        return Rate.create(magnitude=100)
+```
+
 ## Copyright
 
 - Copyright © 2024,2025 Moritz E. Beber.

{httpx_limiter-0.2.0 → httpx_limiter-0.3.0}/README.md

@@ -17,12 +17,16 @@ Install it, for example, with
 pip install httpx-limiter
 ```
 
-## Usage
+## Tutorial
 
-For situations when you need to make a large number of asynchronous request with
-a controlled number of requests per unit time, you can apply rate limiting to an
-HTTPX client using the provided transport. If you want to be able to make ten
-requests per second, for example, use the following:
+You can limit the number of requests made by an HTTPX client using the transports
+provided in this package. That is useful in situations when you need to make a large
+number of asynchronous requests against endpoints that implement a rate limit,
+
+### Single Rate Limit
+
+The simplest use case is to apply a single rate limit to all requests. If you want to be
+able to make twenty requests per second, for example, use the following code:
 
 ```python
 import httpx
@@ -30,11 +34,77 @@ from httpx_limiter import AsyncRateLimitedTransport, Rate
 
 async def main():
     async with httpx.AsyncClient(
-        transport=AsyncRateLimitedTransport.create(rate=Rate.create(magnitude=10)),
+        transport=AsyncRateLimitedTransport.create(rate=Rate.create(magnitude=20)),
     ) as client:
         response = await client.get("https://httpbin.org")
 ```
 
+> [!IMPORTANT]
+> Due to limitations in the design of the underlying leaky bucket implementation,
+> which is used to implement the rate limiting, the magnitude of the rate is also the
+> maximum capacity of the bucket. That means, if you set a rate that is larger than one,
+> a burst of requests equal to that capacity will be allowed. If you do not want to
+> allow any bursts, set the magnitude to one, but the duration to the inverse of your
+> desired rate. If you want to allow twenty requests per second, for example, set the
+> magnitude to 1 and the duration to 0.05 seconds.
+
+### Multiple Rate Limits
+
+For more advanced use cases, you can apply different rate limits based on a concrete
+implementation of the `AbstractRateLimiterRepository`. There are two relevant methods
+that both get passed the current request. One method needs to identify which rate limit
+to apply, and the other method sets the rate limit itself. See the following example:
+
+```python
+import httpx
+from httpx_limiter import AbstractRateLimiterRepository, AsyncMultiRateLimitedTransport, Rate
+
+class DomainBasedRateLimiterRepository(AbstractRateLimiterRepository):
+    """Apply different rate limits based on the domain being requested."""
+    
+    def get_identifier(self, request: httpx.Request) -> str:
+        """Return the domain as the identifier for rate limiting."""
+        return request.url.host
+    
+    def get_rate(self, request: httpx.Request) -> Rate:
+        """Apply the same, but independent rate limit to each domain."""
+        return Rate.create(magnitude=25)
+
+client = httpx.AsyncClient(
+    transport=AsyncMultiRateLimitedTransport.create(
+        repository=DomainBasedRateLimiterRepository(),
+    ),
+)
+```
+
+> [!NOTE]
+> You are free to ignore the request argument and use global information like the time
+> of day to determine the rate limit.
+
+```python
+from datetime import datetime, timezone
+
+import httpx
+from httpx_limiter import AbstractRateLimiterRepository, AsyncMultiRateLimitedTransport, Rate
+
+class DayNightRateLimiterRepository(AbstractRateLimiterRepository):
+    """Apply different rate limits based on the time of day."""
+
+    def get_identifier(self, _: httpx.Request) -> str:
+        """Identify whether it is currently day or night."""
+        if 6 <= datetime.now(tz=timezone.utc).hour < 18:
+            return "day"
+        
+        return "night"
+
+    def get_rate(self, _: httpx.Request) -> Rate:
+        """Apply different rate limits during the day or night."""
+        if self.get_identifier(_) == "day":
+            return Rate.create(magnitude=10)
+        
+        return Rate.create(magnitude=100)
+```
+
 ## Copyright
 
 - Copyright © 2024,2025 Moritz E. Beber.

{httpx_limiter-0.2.0 → httpx_limiter-0.3.0}/pyproject.toml

@@ -161,12 +161,7 @@ ignore_missing_imports = true
 installer = "uv"
 
 [tool.hatch.envs.dev]
-description = """
-Development environment.
-
-Defining a path is a workaround for PyCharm,
-see https://youtrack.jetbrains.com/issue/PY-60410.
-"""
+description = """Development environment."""
 extra-dependencies = [
     "anyio",
     "pytest",
@@ -174,7 +169,6 @@ extra-dependencies = [
     "pytest-watcher",
     "ruff",
 ]
-path = ".venv"
 
 [tool.hatch.envs.style]
 description = """Check the style of the codebase."""
@@ -256,7 +250,6 @@ run = "pytest {args:--cov=httpx_limiter --cov-report=term-missing} {root}/tests/
 description = """Run the acceptance tests."""
 extra-dependencies = [
     "anyio",
-    # TODO: Use behave in future.
     "pytest",
     "pytest-cov",
     "pytest-httpx",

{httpx_limiter-0.2.0 → httpx_limiter-0.3.0}/src/httpx_limiter/__init__.py

@@ -17,3 +17,5 @@
 
 from .rate import Number, Rate
 from .async_rate_limited_transport import AsyncRateLimitedTransport
+from .abstract_rate_limiter_repository import AbstractRateLimiterRepository
+from .async_multi_rate_limited_transport import AsyncMultiRateLimitedTransport

{httpx_limiter-0.2.0 → httpx_limiter-0.3.0}/src/httpx_limiter/_version.py

@@ -17,5 +17,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.2.0'
-__version_tuple__ = version_tuple = (0, 2, 0)
+__version__ = version = '0.3.0'
+__version_tuple__ = version_tuple = (0, 3, 0)

httpx_limiter-0.3.0/src/httpx_limiter/abstract_rate_limiter_repository.py

@@ -0,0 +1,53 @@
+# Copyright (c) 2025 Moritz E. Beber
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may not
+# use this file except in compliance with the License. You may obtain a copy of
+# the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations under
+# the License.
+
+
+"""Provide an abstract repository for rate limiters."""
+
+from abc import ABC, abstractmethod
+
+import httpx
+from aiolimiter import AsyncLimiter
+
+from .rate import Rate
+
+
+class AbstractRateLimiterRepository(ABC):
+    """Define the abstract repository for rate limiters."""
+
+    def __init__(self, **kwargs) -> None:
+        super().__init__(**kwargs)
+        self._limiters: dict[str, AsyncLimiter] = {}
+
+    @abstractmethod
+    def get_identifier(self, request: httpx.Request) -> str:
+        """Return a request-specific identifier."""
+
+    @abstractmethod
+    def get_rate(self, request: httpx.Request) -> Rate:
+        """Return a request-specific rate."""
+
+    def get(self, request: httpx.Request) -> AsyncLimiter:
+        """Return a request-specific rate limiter."""
+        identifier = self.get_identifier(request)
+
+        if identifier not in self._limiters:
+            rate = self.get_rate(request)
+            self._limiters[identifier] = AsyncLimiter(
+                max_rate=rate.magnitude,
+                time_period=rate.in_seconds(),
+            )
+
+        return self._limiters[identifier]
+

httpx_limiter-0.3.0/src/httpx_limiter/async_multi_rate_limited_transport.py

@@ -0,0 +1,84 @@
+# Copyright (c) 2025 Moritz E. Beber
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may not
+# use this file except in compliance with the License. You may obtain a copy of
+# the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations under
+# the License.
+
+
+"""Provide an asynchronous multiple rate-limited transport."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+import httpx
+
+
+if TYPE_CHECKING:  # pragma: no cover
+    from .abstract_rate_limiter_repository import AbstractRateLimiterRepository
+
+
+class AsyncMultiRateLimitedTransport(httpx.AsyncBaseTransport):
+    """
+    Define the asynchronous multiple rate-limited transport.
+
+    This transport consists of a composed transport for handling requests and a
+    repository for rate limiters that are selected based on the request.
+
+    """
+
+    def __init__(
+        self,
+        *,
+        repository: AbstractRateLimiterRepository,
+        transport: httpx.AsyncBaseTransport,
+        **kwargs,
+    ) -> None:
+        super().__init__(**kwargs)
+        self._repo = repository
+        self._transport = transport
+
+    @classmethod
+    def create(
+        cls,
+        *,
+        repository: AbstractRateLimiterRepository,
+        **kwargs: dict,
+    ) -> AsyncMultiRateLimitedTransport:
+        """
+        Create an instance of an asynchronous multiple rate-limited transport.
+
+        This factory method constructs the instance with an underlying
+        `httpx.AsyncHTTPTransport`.
+        That transport is passed any additional keyword arguments.
+
+        Args:
+            repository: The repository to use for rate limiters.
+            **kwargs: Additional keyword arguments are used in the construction of an
+                `httpx.AsyncHTTPTransport`.
+
+        Returns:
+            A default instance of the class created from the given arguments.
+
+        """
+        return cls(
+            repository=repository,
+            transport=httpx.AsyncHTTPTransport(**kwargs),  # type: ignore[arg-type]
+        )
+
+    async def handle_async_request(
+        self,
+        request: httpx.Request,
+    ) -> httpx.Response:
+        """Handle an asynchronous request with rate limiting."""
+        limiter = self._repo.get(request)
+        async with limiter:
+            return await self._transport.handle_async_request(request)

{httpx_limiter-0.2.0 → httpx_limiter-0.3.0}/src/httpx_limiter/async_rate_limited_transport.py

@@ -23,7 +23,7 @@ import httpx
 from aiolimiter import AsyncLimiter
 
 
-if TYPE_CHECKING:
+if TYPE_CHECKING:  # pragma: no cover
     from .rate import Rate