httpx-limiter 0.1.0__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (13) hide show
  1. {httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/PKG-INFO +90 -17
  2. httpx_limiter-0.3.0/README.md +111 -0
  3. {httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/pyproject.toml +40 -30
  4. {httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/src/httpx_limiter/__init__.py +3 -0
  5. {httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/src/httpx_limiter/_version.py +9 -4
  6. httpx_limiter-0.3.0/src/httpx_limiter/abstract_rate_limiter_repository.py +53 -0
  7. httpx_limiter-0.3.0/src/httpx_limiter/async_multi_rate_limited_transport.py +84 -0
  8. {httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/src/httpx_limiter/async_rate_limited_transport.py +16 -9
  9. httpx_limiter-0.3.0/src/httpx_limiter/rate.py +45 -0
  10. httpx_limiter-0.1.0/README.md +0 -39
  11. {httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/.gitignore +0 -0
  12. {httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/LICENSE +0 -0
  13. {httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/src/httpx_limiter/py.typed +0 -0
{httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/PKG-INFO RENAMED
@@ -1,6 +1,6 @@
1
- Metadata-Version: 2.3
1
+ Metadata-Version: 2.4
2
2
  Name: httpx-limiter
3
- Version: 0.1.0
3
+ Version: 0.3.0
4
4
  Summary: A lightweight package that provides rate-limited httpx transports.
5
5
  Project-URL: Homepage, https://github.com/Midnighter/httpx-limiter
6
6
  Project-URL: Documentation, https://httpx-limiter.readthedocs.io
@@ -8,8 +8,9 @@ Project-URL: Source Code, https://github.com/Midnighter/httpx-limiter
8
8
  Project-URL: Bug Tracker, https://github.com/Midnighter/httpx-limiter/issues
9
9
  Project-URL: Download, https://pypi.org/project/httpx-limiter/#files
10
10
  Author-email: "Moritz E. Beber" <midnighter@posteo.net>
11
- License: Apache-2.0
12
- Keywords: httpx,limiter,rate-limit
11
+ License-Expression: Apache-2.0
12
+ License-File: LICENSE
13
+ Keywords: httpx,leaky bucket,limiter,rate-limit
13
14
  Classifier: Development Status :: 2 - Pre-Alpha
14
15
  Classifier: Environment :: Web Environment
15
16
  Classifier: Framework :: AsyncIO
@@ -25,8 +26,8 @@ Classifier: Programming Language :: Python :: 3.13
25
26
  Classifier: Topic :: Internet :: WWW/HTTP
26
27
  Classifier: Typing :: Typed
27
28
  Requires-Python: >=3.10
29
+ Requires-Dist: aiolimiter~=1.2
28
30
  Requires-Dist: httpx~=0.25
29
- Requires-Dist: limiter~=0.5
30
31
  Description-Content-Type: text/markdown
31
32
 
32
33
  # HTTPX Limiter
@@ -41,30 +42,102 @@ _A lightweight package that provides rate-limited httpx transports._
41
42
 
42
43
  ## Installation
43
44
 
44
- The package is published on [PyPI](). Install it, for example, with
45
+ The package is published on [PyPI](https://pypi.org/project/httpx-limiter/).
46
+ Install it, for example, with
45
47
 
46
48
  ```sh
47
49
  pip install httpx-limiter
48
50
  ```
49
51
 
50
- ## Usage
52
+ ## Tutorial
51
53
 
52
- For situations when you need to make a large number of asynchronous request with a
53
- controlled number of requests per time, you can apply rate limiting to an HTTPX client
54
- using the provided transport. If you want to be able to make ten requests per second,
55
- for example, use the following:
54
+ You can limit the number of requests made by an HTTPX client using the transports
55
+ provided in this package. That is useful in situations when you need to make a large
56
+ number of asynchronous requests against endpoints that implement a rate limit,
57
+
58
+ ### Single Rate Limit
59
+
60
+ The simplest use case is to apply a single rate limit to all requests. If you want to be
61
+ able to make twenty requests per second, for example, use the following code:
62
+
63
+ ```python
64
+ import httpx
65
+ from httpx_limiter import AsyncRateLimitedTransport, Rate
66
+
67
+ async def main():
68
+ async with httpx.AsyncClient(
69
+ transport=AsyncRateLimitedTransport.create(rate=Rate.create(magnitude=20)),
70
+ ) as client:
71
+ response = await client.get("https://httpbin.org")
72
+ ```
73
+
74
+ > [!IMPORTANT]
75
+ > Due to limitations in the design of the underlying leaky bucket implementation,
76
+ > which is used to implement the rate limiting, the magnitude of the rate is also the
77
+ > maximum capacity of the bucket. That means, if you set a rate that is larger than one,
78
+ > a burst of requests equal to that capacity will be allowed. If you do not want to
79
+ > allow any bursts, set the magnitude to one, but the duration to the inverse of your
80
+ > desired rate. If you want to allow twenty requests per second, for example, set the
81
+ > magnitude to 1 and the duration to 0.05 seconds.
82
+
83
+ ### Multiple Rate Limits
84
+
85
+ For more advanced use cases, you can apply different rate limits based on a concrete
86
+ implementation of the `AbstractRateLimiterRepository`. There are two relevant methods
87
+ that both get passed the current request. One method needs to identify which rate limit
88
+ to apply, and the other method sets the rate limit itself. See the following example:
56
89
 
57
90
  ```python
58
91
  import httpx
59
- from httpx_limiter import AsyncRateLimitedTransport
92
+ from httpx_limiter import AbstractRateLimiterRepository, AsyncMultiRateLimitedTransport, Rate
93
+
94
+ class DomainBasedRateLimiterRepository(AbstractRateLimiterRepository):
95
+ """Apply different rate limits based on the domain being requested."""
96
+
97
+ def get_identifier(self, request: httpx.Request) -> str:
98
+ """Return the domain as the identifier for rate limiting."""
99
+ return request.url.host
100
+
101
+ def get_rate(self, request: httpx.Request) -> Rate:
102
+ """Apply the same, but independent rate limit to each domain."""
103
+ return Rate.create(magnitude=25)
104
+
105
+ client = httpx.AsyncClient(
106
+ transport=AsyncMultiRateLimitedTransport.create(
107
+ repository=DomainBasedRateLimiterRepository(),
108
+ ),
109
+ )
110
+ ```
111
+
112
+ > [!NOTE]
113
+ > You are free to ignore the request argument and use global information like the time
114
+ > of day to determine the rate limit.
115
+
116
+ ```python
117
+ from datetime import datetime, timezone
118
+
119
+ import httpx
120
+ from httpx_limiter import AbstractRateLimiterRepository, AsyncMultiRateLimitedTransport, Rate
121
+
122
+ class DayNightRateLimiterRepository(AbstractRateLimiterRepository):
123
+ """Apply different rate limits based on the time of day."""
124
+
125
+ def get_identifier(self, _: httpx.Request) -> str:
126
+ """Identify whether it is currently day or night."""
127
+ if 6 <= datetime.now(tz=timezone.utc).hour < 18:
128
+ return "day"
129
+
130
+ return "night"
60
131
 
61
- async with httpx.AsyncClient(
62
- transport=AsyncRateLimitedTransport.create(rate=10, capacity=10),
63
- ) as client:
64
- ...
132
+ def get_rate(self, _: httpx.Request) -> Rate:
133
+ """Apply different rate limits during the day or night."""
134
+ if self.get_identifier(_) == "day":
135
+ return Rate.create(magnitude=10)
136
+
137
+ return Rate.create(magnitude=100)
65
138
  ```
66
139
 
67
140
  ## Copyright
68
141
 
69
- - Copyright © 2024 Moritz E. Beber.
142
+ - Copyright © 2024,2025 Moritz E. Beber.
70
143
  - Free software distributed under the [Apache Software License 2.0](./LICENSE).
httpx_limiter-0.3.0/README.md ADDED
@@ -0,0 +1,111 @@
1
+ # HTTPX Limiter
2
+
3
+ | | |
4
+ |---|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
5
+ | Package | [![Latest PyPI Version](https://img.shields.io/pypi/v/httpx-limiter.svg)](https://pypi.org/project/httpx-limiter/) [![Supported Python Versions](https://img.shields.io/pypi/pyversions/httpx-limiter.svg)](https://pypi.org/project/httpx-limiter/) [![Documentation](https://readthedocs.org/projects/httpx-limiter/badge/?version=latest)](https://httpx-limiter.readthedocs.io/en/latest/?badge=latest) |
6
+ | Meta | [![Apache-2.0](https://img.shields.io/pypi/l/httpx-limiter.svg)](LICENSE) [![Code of Conduct](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](.github/CODE_OF_CONDUCT.md) [![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/) [![Code Style Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black) [![Linting: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) |
7
+ | Automation | |
8
+
9
+ _A lightweight package that provides rate-limited httpx transports._
10
+
11
+ ## Installation
12
+
13
+ The package is published on [PyPI](https://pypi.org/project/httpx-limiter/).
14
+ Install it, for example, with
15
+
16
+ ```sh
17
+ pip install httpx-limiter
18
+ ```
19
+
20
+ ## Tutorial
21
+
22
+ You can limit the number of requests made by an HTTPX client using the transports
23
+ provided in this package. That is useful in situations when you need to make a large
24
+ number of asynchronous requests against endpoints that implement a rate limit,
25
+
26
+ ### Single Rate Limit
27
+
28
+ The simplest use case is to apply a single rate limit to all requests. If you want to be
29
+ able to make twenty requests per second, for example, use the following code:
30
+
31
+ ```python
32
+ import httpx
33
+ from httpx_limiter import AsyncRateLimitedTransport, Rate
34
+
35
+ async def main():
36
+ async with httpx.AsyncClient(
37
+ transport=AsyncRateLimitedTransport.create(rate=Rate.create(magnitude=20)),
38
+ ) as client:
39
+ response = await client.get("https://httpbin.org")
40
+ ```
41
+
42
+ > [!IMPORTANT]
43
+ > Due to limitations in the design of the underlying leaky bucket implementation,
44
+ > which is used to implement the rate limiting, the magnitude of the rate is also the
45
+ > maximum capacity of the bucket. That means, if you set a rate that is larger than one,
46
+ > a burst of requests equal to that capacity will be allowed. If you do not want to
47
+ > allow any bursts, set the magnitude to one, but the duration to the inverse of your
48
+ > desired rate. If you want to allow twenty requests per second, for example, set the
49
+ > magnitude to 1 and the duration to 0.05 seconds.
50
+
51
+ ### Multiple Rate Limits
52
+
53
+ For more advanced use cases, you can apply different rate limits based on a concrete
54
+ implementation of the `AbstractRateLimiterRepository`. There are two relevant methods
55
+ that both get passed the current request. One method needs to identify which rate limit
56
+ to apply, and the other method sets the rate limit itself. See the following example:
57
+
58
+ ```python
59
+ import httpx
60
+ from httpx_limiter import AbstractRateLimiterRepository, AsyncMultiRateLimitedTransport, Rate
61
+
62
+ class DomainBasedRateLimiterRepository(AbstractRateLimiterRepository):
63
+ """Apply different rate limits based on the domain being requested."""
64
+
65
+ def get_identifier(self, request: httpx.Request) -> str:
66
+ """Return the domain as the identifier for rate limiting."""
67
+ return request.url.host
68
+
69
+ def get_rate(self, request: httpx.Request) -> Rate:
70
+ """Apply the same, but independent rate limit to each domain."""
71
+ return Rate.create(magnitude=25)
72
+
73
+ client = httpx.AsyncClient(
74
+ transport=AsyncMultiRateLimitedTransport.create(
75
+ repository=DomainBasedRateLimiterRepository(),
76
+ ),
77
+ )
78
+ ```
79
+
80
+ > [!NOTE]
81
+ > You are free to ignore the request argument and use global information like the time
82
+ > of day to determine the rate limit.
83
+
84
+ ```python
85
+ from datetime import datetime, timezone
86
+
87
+ import httpx
88
+ from httpx_limiter import AbstractRateLimiterRepository, AsyncMultiRateLimitedTransport, Rate
89
+
90
+ class DayNightRateLimiterRepository(AbstractRateLimiterRepository):
91
+ """Apply different rate limits based on the time of day."""
92
+
93
+ def get_identifier(self, _: httpx.Request) -> str:
94
+ """Identify whether it is currently day or night."""
95
+ if 6 <= datetime.now(tz=timezone.utc).hour < 18:
96
+ return "day"
97
+
98
+ return "night"
99
+
100
+ def get_rate(self, _: httpx.Request) -> Rate:
101
+ """Apply different rate limits during the day or night."""
102
+ if self.get_identifier(_) == "day":
103
+ return Rate.create(magnitude=10)
104
+
105
+ return Rate.create(magnitude=100)
106
+ ```
107
+
108
+ ## Copyright
109
+
110
+ - Copyright © 2024,2025 Moritz E. Beber.
111
+ - Free software distributed under the [Apache Software License 2.0](./LICENSE).
{httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/pyproject.toml RENAMED
@@ -41,10 +41,11 @@ keywords = [
41
41
  "httpx",
42
42
  "limiter",
43
43
  "rate-limit",
44
+ "leaky bucket",
44
45
  ]
45
46
  dependencies = [
46
47
  "httpx ~=0.25",
47
- "limiter ~=0.5",
48
+ "aiolimiter ~=1.2",
48
49
  ]
49
50
 
50
51
  [project.urls]
@@ -103,8 +104,6 @@ ignore = [
103
104
  "D407", # 'Missing dashed underline after section' ignored because Google style docstrings don't underline.
104
105
  "ANN002", # 'Missing type annotation for {*args} in method'.
105
106
  "ANN003", # 'Missing type annotation for {*kwargs} in method'.
106
- "ANN101", # 'Missing type annotation for {self} in method'.
107
- "ANN102", # 'Missing type annotation for {cls} in classmethod'.
108
107
  ]
109
108
  exclude = [
110
109
  "src/httpx_limiter/_version.py",
@@ -130,7 +129,7 @@ known-first-party = ["src", "httpx_limiter"]
130
129
  lines-after-imports = 2
131
130
 
132
131
  [tool.pydoclint]
133
- style = "google" # TODO: Other styles are possible here, like 'numpy'
132
+ style = "google"
134
133
  arg-type-hints-in-docstring = false
135
134
  check-return-types = false
136
135
  check-yield-types = false
@@ -162,31 +161,26 @@ ignore_missing_imports = true
162
161
  installer = "uv"
163
162
 
164
163
  [tool.hatch.envs.dev]
165
- description = """
166
- Development environment.
167
-
168
- Defining a path is a workaround for PyCharm,
169
- see https://youtrack.jetbrains.com/issue/PY-60410.
170
- """
164
+ description = """Development environment."""
171
165
  extra-dependencies = [
172
166
  "anyio",
173
167
  "pytest",
174
168
  "pytest-httpx",
175
169
  "pytest-watcher",
170
+ "ruff",
176
171
  ]
177
- path = ".venv"
178
172
 
179
173
  [tool.hatch.envs.style]
180
174
  description = """Check the style of the codebase."""
181
175
  dependencies = [
182
- "pydoclint",
176
+ "pydoclint ~=0.6",
183
177
  "ruff",
184
178
  ]
185
179
  detached = true
186
180
  installer = "uv"
187
181
 
188
182
  [tool.hatch.envs.style.scripts]
189
- docstrings = "pydoclint"
183
+ docstrings = ["pydoclint src", "pydoclint tests"]
190
184
  code = "ruff check {args}"
191
185
  format = "ruff format {args}"
192
186
  check = ["docstrings", "code"]
@@ -221,23 +215,39 @@ extra-dependencies = [
221
215
  build = "mkdocs build {args:--clean --strict}"
222
216
  serve = "mkdocs serve {args}"
223
217
 
224
- [tool.hatch.envs.install]
225
- description = """Test the installation the package."""
226
- dependencies = [
227
- "pip",
228
- "twine",
229
- ]
230
- detached = true
218
+ #[tool.hatch.envs.install]
219
+ #description = """Test the installation the package."""
220
+ #dependencies = [
221
+ # "pip",
222
+ # "twine",
223
+ #]
224
+ #detached = true
225
+
226
+ #[tool.hatch.envs.install.scripts]
227
+ #check = [
228
+ # "pip check",
229
+ # "hatch build {args:--clean}",
230
+ # "twine check dist/*",
231
+ #]
231
232
 
232
- [tool.hatch.envs.install.scripts]
233
- check = [
234
- "pip check",
235
- "hatch build {args:--clean}",
236
- "twine check dist/*",
233
+ [tool.hatch.envs.unit-tests]
234
+ description = """Run the unit tests."""
235
+ extra-dependencies = [
236
+ "anyio",
237
+ "pytest",
238
+ "pytest-cov",
239
+ "pytest-httpx",
240
+ "pytest-raises",
237
241
  ]
238
242
 
239
- [tool.hatch.envs.test]
240
- description = """Run the test suite."""
243
+ [[tool.hatch.envs.unit-tests.matrix]]
244
+ python = ["3.10", "3.11", "3.12", "3.13"]
245
+
246
+ [tool.hatch.envs.unit-tests.scripts]
247
+ run = "pytest {args:--cov=httpx_limiter --cov-report=term-missing} {root}/tests/unit"
248
+
249
+ [tool.hatch.envs.acceptance-tests]
250
+ description = """Run the acceptance tests."""
241
251
  extra-dependencies = [
242
252
  "anyio",
243
253
  "pytest",
@@ -246,8 +256,8 @@ extra-dependencies = [
246
256
  "pytest-raises",
247
257
  ]
248
258
 
249
- [[tool.hatch.envs.test.matrix]]
259
+ [[tool.hatch.envs.acceptance-tests.matrix]]
250
260
  python = ["3.10", "3.11", "3.12", "3.13"]
251
261
 
252
- [tool.hatch.envs.test.scripts]
253
- run = "pytest {args:--cov=httpx_limiter --cov-report=term-missing}"
262
+ [tool.hatch.envs.acceptance-tests.scripts]
263
+ run = "pytest {args:--cov=httpx_limiter --cov-report=term-missing} {root}/tests/acceptance"
{httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/src/httpx_limiter/__init__.py RENAMED
@@ -15,4 +15,7 @@
15
15
 
16
16
  """Provide top level symbols."""
17
17
 
18
+ from .rate import Number, Rate
18
19
  from .async_rate_limited_transport import AsyncRateLimitedTransport
20
+ from .abstract_rate_limiter_repository import AbstractRateLimiterRepository
21
+ from .async_multi_rate_limited_transport import AsyncMultiRateLimitedTransport
{httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/src/httpx_limiter/_version.py RENAMED
@@ -1,8 +1,13 @@
1
- # file generated by setuptools_scm
1
+ # file generated by setuptools-scm
2
2
  # don't change, don't track in version control
3
+
4
+ __all__ = ["__version__", "__version_tuple__", "version", "version_tuple"]
5
+
3
6
  TYPE_CHECKING = False
4
7
  if TYPE_CHECKING:
5
- from typing import Tuple, Union
8
+ from typing import Tuple
9
+ from typing import Union
10
+
6
11
  VERSION_TUPLE = Tuple[Union[int, str], ...]
7
12
  else:
8
13
  VERSION_TUPLE = object
@@ -12,5 +17,5 @@ __version__: str
12
17
  __version_tuple__: VERSION_TUPLE
13
18
  version_tuple: VERSION_TUPLE
14
19
 
15
- __version__ = version = '0.1.0'
16
- __version_tuple__ = version_tuple = (0, 1, 0)
20
+ __version__ = version = '0.3.0'
21
+ __version_tuple__ = version_tuple = (0, 3, 0)
httpx_limiter-0.3.0/src/httpx_limiter/abstract_rate_limiter_repository.py ADDED
@@ -0,0 +1,53 @@
1
+ # Copyright (c) 2025 Moritz E. Beber
2
+ #
3
+ # Licensed under the Apache License, Version 2.0 (the "License"); you may not
4
+ # use this file except in compliance with the License. You may obtain a copy of
5
+ # the License at
6
+ #
7
+ # https://www.apache.org/licenses/LICENSE-2.0
8
+ #
9
+ # Unless required by applicable law or agreed to in writing, software
10
+ # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
11
+ # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
12
+ # License for the specific language governing permissions and limitations under
13
+ # the License.
14
+
15
+
16
+ """Provide an abstract repository for rate limiters."""
17
+
18
+ from abc import ABC, abstractmethod
19
+
20
+ import httpx
21
+ from aiolimiter import AsyncLimiter
22
+
23
+ from .rate import Rate
24
+
25
+
26
+ class AbstractRateLimiterRepository(ABC):
27
+ """Define the abstract repository for rate limiters."""
28
+
29
+ def __init__(self, **kwargs) -> None:
30
+ super().__init__(**kwargs)
31
+ self._limiters: dict[str, AsyncLimiter] = {}
32
+
33
+ @abstractmethod
34
+ def get_identifier(self, request: httpx.Request) -> str:
35
+ """Return a request-specific identifier."""
36
+
37
+ @abstractmethod
38
+ def get_rate(self, request: httpx.Request) -> Rate:
39
+ """Return a request-specific rate."""
40
+
41
+ def get(self, request: httpx.Request) -> AsyncLimiter:
42
+ """Return a request-specific rate limiter."""
43
+ identifier = self.get_identifier(request)
44
+
45
+ if identifier not in self._limiters:
46
+ rate = self.get_rate(request)
47
+ self._limiters[identifier] = AsyncLimiter(
48
+ max_rate=rate.magnitude,
49
+ time_period=rate.in_seconds(),
50
+ )
51
+
52
+ return self._limiters[identifier]
53
+
httpx_limiter-0.3.0/src/httpx_limiter/async_multi_rate_limited_transport.py ADDED
@@ -0,0 +1,84 @@
1
+ # Copyright (c) 2025 Moritz E. Beber
2
+ #
3
+ # Licensed under the Apache License, Version 2.0 (the "License"); you may not
4
+ # use this file except in compliance with the License. You may obtain a copy of
5
+ # the License at
6
+ #
7
+ # https://www.apache.org/licenses/LICENSE-2.0
8
+ #
9
+ # Unless required by applicable law or agreed to in writing, software
10
+ # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
11
+ # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
12
+ # License for the specific language governing permissions and limitations under
13
+ # the License.
14
+
15
+
16
+ """Provide an asynchronous multiple rate-limited transport."""
17
+
18
+ from __future__ import annotations
19
+
20
+ from typing import TYPE_CHECKING
21
+
22
+ import httpx
23
+
24
+
25
+ if TYPE_CHECKING: # pragma: no cover
26
+ from .abstract_rate_limiter_repository import AbstractRateLimiterRepository
27
+
28
+
29
+ class AsyncMultiRateLimitedTransport(httpx.AsyncBaseTransport):
30
+ """
31
+ Define the asynchronous multiple rate-limited transport.
32
+
33
+ This transport consists of a composed transport for handling requests and a
34
+ repository for rate limiters that are selected based on the request.
35
+
36
+ """
37
+
38
+ def __init__(
39
+ self,
40
+ *,
41
+ repository: AbstractRateLimiterRepository,
42
+ transport: httpx.AsyncBaseTransport,
43
+ **kwargs,
44
+ ) -> None:
45
+ super().__init__(**kwargs)
46
+ self._repo = repository
47
+ self._transport = transport
48
+
49
+ @classmethod
50
+ def create(
51
+ cls,
52
+ *,
53
+ repository: AbstractRateLimiterRepository,
54
+ **kwargs: dict,
55
+ ) -> AsyncMultiRateLimitedTransport:
56
+ """
57
+ Create an instance of an asynchronous multiple rate-limited transport.
58
+
59
+ This factory method constructs the instance with an underlying
60
+ `httpx.AsyncHTTPTransport`.
61
+ That transport is passed any additional keyword arguments.
62
+
63
+ Args:
64
+ repository: The repository to use for rate limiters.
65
+ **kwargs: Additional keyword arguments are used in the construction of an
66
+ `httpx.AsyncHTTPTransport`.
67
+
68
+ Returns:
69
+ A default instance of the class created from the given arguments.
70
+
71
+ """
72
+ return cls(
73
+ repository=repository,
74
+ transport=httpx.AsyncHTTPTransport(**kwargs), # type: ignore[arg-type]
75
+ )
76
+
77
+ async def handle_async_request(
78
+ self,
79
+ request: httpx.Request,
80
+ ) -> httpx.Response:
81
+ """Handle an asynchronous request with rate limiting."""
82
+ limiter = self._repo.get(request)
83
+ async with limiter:
84
+ return await self._transport.handle_async_request(request)
{httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/src/httpx_limiter/async_rate_limited_transport.py RENAMED
@@ -17,8 +17,14 @@
17
17
 
18
18
  from __future__ import annotations
19
19
 
20
+ from typing import TYPE_CHECKING
21
+
20
22
  import httpx
21
- from limiter import Limiter, Tokens
23
+ from aiolimiter import AsyncLimiter
24
+
25
+
26
+ if TYPE_CHECKING: # pragma: no cover
27
+ from .rate import Rate
22
28
 
23
29
 
24
30
  class AsyncRateLimitedTransport(httpx.AsyncBaseTransport):
@@ -26,7 +32,7 @@ class AsyncRateLimitedTransport(httpx.AsyncBaseTransport):
26
32
  Define the asynchronous rate-limited transport.
27
33
 
28
34
  This transport consists of a composed transport for handling requests and an
29
- implementation of a token bucket algorithm in order to rate-limit the number of
35
+ implementation of a leaky bucket algorithm in order to rate-limit the number of
30
36
  requests.
31
37
 
32
38
  """
@@ -34,7 +40,7 @@ class AsyncRateLimitedTransport(httpx.AsyncBaseTransport):
34
40
  def __init__(
35
41
  self,
36
42
  *,
37
- limiter: Limiter,
43
+ limiter: AsyncLimiter,
38
44
  transport: httpx.AsyncBaseTransport,
39
45
  **kwargs,
40
46
  ) -> None:
@@ -46,20 +52,18 @@ class AsyncRateLimitedTransport(httpx.AsyncBaseTransport):
46
52
  def create(
47
53
  cls,
48
54
  *,
49
- rate: Tokens,
50
- capacity: Tokens,
55
+ rate: Rate,
51
56
  **kwargs: dict,
52
57
  ) -> AsyncRateLimitedTransport:
53
58
  """
54
- Create an instance of asynchronous limited transport.
59
+ Create an instance of asynchronous rate-limited transport.
55
60
 
56
61
  This factory method constructs the instance with an underlying
57
62
  `httpx.AsyncHTTPTransport`.
58
63
  That transport is passed any additional keyword arguments.
59
64
 
60
65
  Args:
61
- rate: The refresh rate per second for bucket tokens.
62
- capacity: The maximum number of tokens the bucket can contain.
66
+ rate: The maximum rate per interval at which bucket capacity is restored.
63
67
  **kwargs: Additional keyword arguments are used in the construction of an
64
68
  `httpx.AsyncHTTPTransport`.
65
69
 
@@ -68,7 +72,10 @@ class AsyncRateLimitedTransport(httpx.AsyncBaseTransport):
68
72
 
69
73
  """
70
74
  return cls(
71
- limiter=Limiter(rate=rate, capacity=capacity),
75
+ limiter=AsyncLimiter(
76
+ max_rate=rate.magnitude,
77
+ time_period=rate.in_seconds(),
78
+ ),
72
79
  transport=httpx.AsyncHTTPTransport(**kwargs), # type: ignore[arg-type]
73
80
  )
74
81
 
httpx_limiter-0.3.0/src/httpx_limiter/rate.py ADDED
@@ -0,0 +1,45 @@
1
+ # Copyright (c) 2025 Moritz E. Beber
2
+ #
3
+ # Licensed under the Apache License, Version 2.0 (the "License"); you may not
4
+ # use this file except in compliance with the License. You may obtain a copy of
5
+ # the License at
6
+ #
7
+ # https://www.apache.org/licenses/LICENSE-2.0
8
+ #
9
+ # Unless required by applicable law or agreed to in writing, software
10
+ # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
11
+ # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
12
+ # License for the specific language governing permissions and limitations under
13
+ # the License.
14
+
15
+
16
+ """Provide a definition of a rate."""
17
+
18
+ from __future__ import annotations
19
+
20
+ from datetime import timedelta
21
+ from typing import NamedTuple
22
+
23
+
24
+ Number = float | int
25
+
26
+
27
+ class Rate(NamedTuple):
28
+ """Define the rate."""
29
+
30
+ magnitude: float
31
+ duration: timedelta
32
+
33
+ @classmethod
34
+ def create(cls, magnitude: Number = 1, duration: timedelta | Number = 1) -> Rate:
35
+ """Create a rate."""
36
+ magnitude = float(magnitude)
37
+
38
+ if not isinstance(duration, timedelta):
39
+ duration = timedelta(seconds=float(duration))
40
+
41
+ return cls(magnitude=magnitude, duration=duration)
42
+
43
+ def in_seconds(self) -> float:
44
+ """Return the duration in unit seconds."""
45
+ return self.duration.total_seconds()
httpx_limiter-0.1.0/README.md DELETED
@@ -1,39 +0,0 @@
1
- # HTTPX Limiter
2
-
3
- | | |
4
- |---|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
5
- | Package | [![Latest PyPI Version](https://img.shields.io/pypi/v/httpx-limiter.svg)](https://pypi.org/project/httpx-limiter/) [![Supported Python Versions](https://img.shields.io/pypi/pyversions/httpx-limiter.svg)](https://pypi.org/project/httpx-limiter/) [![Documentation](https://readthedocs.org/projects/httpx-limiter/badge/?version=latest)](https://httpx-limiter.readthedocs.io/en/latest/?badge=latest) |
6
- | Meta | [![Apache-2.0](https://img.shields.io/pypi/l/httpx-limiter.svg)](LICENSE) [![Code of Conduct](https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg)](.github/CODE_OF_CONDUCT.md) [![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/) [![Code Style Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black) [![Linting: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) |
7
- | Automation | |
8
-
9
- _A lightweight package that provides rate-limited httpx transports._
10
-
11
- ## Installation
12
-
13
- The package is published on [PyPI](). Install it, for example, with
14
-
15
- ```sh
16
- pip install httpx-limiter
17
- ```
18
-
19
- ## Usage
20
-
21
- For situations when you need to make a large number of asynchronous request with a
22
- controlled number of requests per time, you can apply rate limiting to an HTTPX client
23
- using the provided transport. If you want to be able to make ten requests per second,
24
- for example, use the following:
25
-
26
- ```python
27
- import httpx
28
- from httpx_limiter import AsyncRateLimitedTransport
29
-
30
- async with httpx.AsyncClient(
31
- transport=AsyncRateLimitedTransport.create(rate=10, capacity=10),
32
- ) as client:
33
- ...
34
- ```
35
-
36
- ## Copyright
37
-
38
- - Copyright © 2024 Moritz E. Beber.
39
- - Free software distributed under the [Apache Software License 2.0](./LICENSE).
{httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/.gitignore RENAMED
File without changes
{httpx_limiter-0.1.0 → httpx_limiter-0.3.0}/LICENSE RENAMED
File without changes