cancellable-http-client 1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- cancellable_http_client-1.0/LICENSE +15 -0
- cancellable_http_client-1.0/PKG-INFO +204 -0
- cancellable_http_client-1.0/README.md +178 -0
- cancellable_http_client-1.0/cancellable_http_client.egg-info/PKG-INFO +204 -0
- cancellable_http_client-1.0/cancellable_http_client.egg-info/SOURCES.txt +9 -0
- cancellable_http_client-1.0/cancellable_http_client.egg-info/dependency_links.txt +1 -0
- cancellable_http_client-1.0/cancellable_http_client.egg-info/top_level.txt +1 -0
- cancellable_http_client-1.0/cancellable_http_client.py +289 -0
- cancellable_http_client-1.0/pyproject.toml +33 -0
- cancellable_http_client-1.0/setup.cfg +4 -0
- cancellable_http_client-1.0/tests/test_cancellable_http_client.py +428 -0
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
Copyright 2026 Sakilabo Corporation Ltd.
|
|
2
|
+
|
|
3
|
+
The Universal Permissive License (UPL), Version 1.0
|
|
4
|
+
|
|
5
|
+
Subject to the condition set forth below, permission is hereby granted to any person obtaining a copy of this software, associated documentation and/or data (collectively the "Software"), free of charge and under any and all copyright rights in the Software, and any and all patent rights owned or freely licensable by each licensor hereunder covering either (i) the unmodified Software as contributed to or provided by such licensor, or (ii) the Larger Works (as defined below), to deal in both
|
|
6
|
+
|
|
7
|
+
(a) the Software, and
|
|
8
|
+
(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if one is included with the Software (each a "Larger Work" to which the Software is contributed by such licensors),
|
|
9
|
+
|
|
10
|
+
without restriction, including without limitation the rights to copy, create derivative works of, display, perform, and distribute the Software and make, use, sell, offer for sale, import, export, have made, and have sold the Software and the Larger Work(s), and to sublicense the foregoing rights on either these or other terms.
|
|
11
|
+
|
|
12
|
+
This license is subject to the following condition:
|
|
13
|
+
The above copyright notice and either this complete permission notice or at a minimum a reference to the UPL must be included in all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: cancellable-http-client
|
|
3
|
+
Version: 1.0
|
|
4
|
+
Summary: A tiny, dependency-free HTTP client for Python with cancellable in-flight requests and hard wall-clock timeout.
|
|
5
|
+
Author: Sakilabo Corporation Ltd.
|
|
6
|
+
License-Expression: UPL-1.0
|
|
7
|
+
Project-URL: Homepage, https://github.com/sakilabo/cancellable-http-client
|
|
8
|
+
Project-URL: Issues, https://github.com/sakilabo/cancellable-http-client/issues
|
|
9
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.7
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
+
Classifier: Topic :: Internet :: WWW/HTTP
|
|
22
|
+
Requires-Python: >=3.7
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
License-File: LICENSE
|
|
25
|
+
Dynamic: license-file
|
|
26
|
+
|
|
27
|
+
# cancellable_http_client
|
|
28
|
+
|
|
29
|
+
A tiny, dependency-free HTTP client for Python with **cancellable in-flight requests** and **hard wall-clock timeout**.
|
|
30
|
+
|
|
31
|
+
- Standard library only — no `requests`, no `httpx`, no `urllib3`.
|
|
32
|
+
- Single file, ~300 lines.
|
|
33
|
+
- Synchronous API that plays well with `threading`-based workers.
|
|
34
|
+
- Safe `close()` from any thread, at any time, including mid-transfer.
|
|
35
|
+
- Hard wall-clock `timeout` that bounds the entire request.
|
|
36
|
+
|
|
37
|
+
## Why it exists
|
|
38
|
+
|
|
39
|
+
Python has no clean way to interrupt a thread that is blocked on a socket read. `concurrent.futures.Future.cancel()` does nothing once the task has started, and `requests` / `urllib.urlopen()` give you no handle to abort an in-flight request.
|
|
40
|
+
|
|
41
|
+
The one primitive that *does* work is closing the underlying socket: any pending `recv()` immediately unblocks with an error. This module wraps that trick behind a tiny, boring API so you don't have to reinvent it — or worry about the lifecycle edge cases — every time you need it.
|
|
42
|
+
|
|
43
|
+
If your codebase is built around `asyncio`, you don't need this; use `httpx` and `task.cancel()` instead. This module targets the very real case where you have existing threaded code and you want one HTTP call in the middle of it to be cancellable, without rewriting everything to be async.
|
|
44
|
+
|
|
45
|
+
## Usage
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
import time
|
|
49
|
+
import cancellable_http_client as client
|
|
50
|
+
|
|
51
|
+
req = client.Request("https://example.com/")
|
|
52
|
+
req.start() # the actual TCP connection happens here
|
|
53
|
+
start = time.monotonic()
|
|
54
|
+
while not req.done:
|
|
55
|
+
if time.monotonic() - start > 5:
|
|
56
|
+
print("taking too long, aborting...")
|
|
57
|
+
req.close() # interrupts the request if it's still in-flight
|
|
58
|
+
req.wait(0.1) # wait a bit before checking again
|
|
59
|
+
if req.error:
|
|
60
|
+
print(f"failed: {req.error}")
|
|
61
|
+
elif req.response and req.response.status == 200:
|
|
62
|
+
print(req.response.body)
|
|
63
|
+
req.close() # safe to call any time, even mid-flight
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
You can also use it as a context manager:
|
|
67
|
+
|
|
68
|
+
```python
|
|
69
|
+
with client.Request("https://example.com/") as req:
|
|
70
|
+
req.start()
|
|
71
|
+
req.wait(timeout=5)
|
|
72
|
+
...
|
|
73
|
+
# close() is called automatically on exit
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Sharing a thread pool
|
|
77
|
+
|
|
78
|
+
By default each `Request` spawns its own daemon thread. To reuse a pool instead, assign an `Executor` to the module-level attribute:
|
|
79
|
+
|
|
80
|
+
```python
|
|
81
|
+
from concurrent.futures import ThreadPoolExecutor
|
|
82
|
+
client.executor = ThreadPoolExecutor(max_workers=8)
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
## API
|
|
86
|
+
|
|
87
|
+
### `Request(url, method="GET", headers=None, body=b"", socket_timeout=30, timeout=None)`
|
|
88
|
+
|
|
89
|
+
Construct a request. No network I/O happens here — connection failures are reported via `error` after `start()`.
|
|
90
|
+
|
|
91
|
+
- **`socket_timeout`** — per-socket-operation timeout in seconds, passed to `http.client.HTTPConnection`.
|
|
92
|
+
- **`timeout`** — wall-clock limit in seconds for the entire request. Triggers `close()` automatically if the request is not done in time. `None` disables.
|
|
93
|
+
- **`start()`** — kick off the request. Non-blocking.
|
|
94
|
+
- **`wait(timeout=None) -> bool`** — block until the request finishes. Returns `True` on completion, `False` on timeout.
|
|
95
|
+
- **`close()`** — abort the request and release resources. Safe to call any time, from any thread, any number of times.
|
|
96
|
+
- **`done`** *(property)* — `True` once the request has finished (success, failure, or close).
|
|
97
|
+
- **`response`** — a `Response` object on success, otherwise `None`.
|
|
98
|
+
- **`error`** — the exception raised during the request, or `None`.
|
|
99
|
+
|
|
100
|
+
### `Response`
|
|
101
|
+
|
|
102
|
+
A read-only, socket-free container exposing the same attributes as `http.client.HTTPResponse`:
|
|
103
|
+
|
|
104
|
+
- `status`, `reason`, `version`
|
|
105
|
+
- `headers` (an `http.client.HTTPMessage`)
|
|
106
|
+
- `body` (`bytes`, eagerly read)
|
|
107
|
+
- `getheader(name, default=None)`, `getheaders()`
|
|
108
|
+
|
|
109
|
+
## Robust timeout
|
|
110
|
+
|
|
111
|
+
Most Python HTTP clients set a *per-socket-operation* timeout (`socket.settimeout`). This leaves several gaps:
|
|
112
|
+
|
|
113
|
+
- **Slow drip** — a server that sends one byte every 29 seconds never triggers a 30-second socket timeout, yet the total transfer can take arbitrarily long.
|
|
114
|
+
- **DNS resolution** — `socket.getaddrinfo()` is a blocking C library call with no timeout parameter. Python cannot interrupt it.
|
|
115
|
+
- **Total elapsed time** — there is no built-in way to cap the wall-clock time of an entire request across connection, TLS handshake, sending, and receiving.
|
|
116
|
+
|
|
117
|
+
`cancellable_http_client` addresses this with two separate knobs:
|
|
118
|
+
|
|
119
|
+
| Parameter | Scope | Default |
|
|
120
|
+
|---|---|---|
|
|
121
|
+
| `socket_timeout` | Per socket operation (connect, send, recv) | 30 s |
|
|
122
|
+
| `timeout` | Wall-clock limit on the entire request | None (no limit) |
|
|
123
|
+
|
|
124
|
+
When `timeout` fires it calls `close()`, which immediately unblocks any pending socket operation by closing the underlying connection. This gives you a hard upper bound on how long `wait()` will block — something that `socket_timeout` alone cannot guarantee.
|
|
125
|
+
|
|
126
|
+
## Comparison with existing libraries
|
|
127
|
+
|
|
128
|
+
| | cancellable_http_client | httpx | requests |
|
|
129
|
+
|---|---|---|---|
|
|
130
|
+
| Cancel an in-flight request from another thread | ✅ | ⚠️ async, [unreliable](https://github.com/encode/httpx/issues/1461) | ⚠️ hacky |
|
|
131
|
+
| Hard wall-clock timeout on entire request | ✅ | ⚠️ per-operation | ⚠️ per-operation |
|
|
132
|
+
| Synchronous API | ✅ | ✅ (also async) | ✅ |
|
|
133
|
+
| No third-party dependencies | ✅ | ❌ | ❌ |
|
|
134
|
+
| Line count | ~300 | thousands | thousands |
|
|
135
|
+
| Fits a threading-based worker | ✅ | ❌ | ⚠️ |
|
|
136
|
+
| Redirects, cookies, User-Agent | ⚠️ manual | ✅ | ✅ |
|
|
137
|
+
|
|
138
|
+
`httpx` is a good choice if you are already in an `asyncio` world, though `task.cancel()` on in-flight requests [can leave the connection pool in a broken state](https://github.com/encode/httpx/issues/1461). `requests` does not offer a reliable way to interrupt an in-flight call; `Session.close()` does not forcibly close active sockets ([psf/requests#5633](https://github.com/psf/requests/issues/5633)).
|
|
139
|
+
|
|
140
|
+
## Limitations
|
|
141
|
+
|
|
142
|
+
This library is a thin wrapper around `http.client` and does not provide the high-level conveniences found in `requests` or `httpx`:
|
|
143
|
+
|
|
144
|
+
- No automatic redirect following
|
|
145
|
+
- No cookie management
|
|
146
|
+
- No default User-Agent header
|
|
147
|
+
- No HTTP/2 support
|
|
148
|
+
|
|
149
|
+
These are all `http.client` limitations, not restrictions added by this library. You can still handle them manually via the `headers` parameter.
|
|
150
|
+
|
|
151
|
+
## Tests
|
|
152
|
+
|
|
153
|
+
```
|
|
154
|
+
python -m unittest discover -s tests -v
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
No third-party test dependencies. Tests use local throwaway servers (normal, slow, blackhole, mid-body disconnect) to exercise cancellation, timeout, and error paths without touching the network.
|
|
158
|
+
|
|
159
|
+
## License
|
|
160
|
+
|
|
161
|
+
Copyright 2026 Sakilabo Corporation Ltd.
|
|
162
|
+
Licensed under the Universal Permissive License v 1.0
|
|
163
|
+
([UPL-1.0](https://oss.oracle.com/licenses/upl/)).
|
|
164
|
+
|
|
165
|
+
---
|
|
166
|
+
|
|
167
|
+
## References (for the eventual public release)
|
|
168
|
+
|
|
169
|
+
Background reading and related work collected during the design of this module. Useful as citations in the README or a launch blog post.
|
|
170
|
+
|
|
171
|
+
### Prior art / closest existing work
|
|
172
|
+
|
|
173
|
+
- **["TIL: Stopping Requests Mid Flight" — haykot.dev](https://haykot.dev/blog/til-stopping-requests-mid-flight/)**
|
|
174
|
+
A blog post describing the "close the socket from another thread" trick as a personal discovery. The same underlying idea as this module, but kept as a snippet rather than packaged as a library.
|
|
175
|
+
|
|
176
|
+
- **[httpcore on PyPI](https://pypi.org/project/httpcore/)**
|
|
177
|
+
the low-level HTTP engine behind `httpx`. Supports cancellation via async task cancellation; relies on `asyncio` or `trio`.
|
|
178
|
+
|
|
179
|
+
- **[HTTPX](https://www.python-httpx.org/)**
|
|
180
|
+
modern high-level HTTP client; cancellation is done via `task.cancel()` in an async context.
|
|
181
|
+
|
|
182
|
+
- **[asyncio-cancel-token](https://asyncio-cancel-token.readthedocs.io/en/latest/cancel_token.html)**
|
|
183
|
+
a cancellation-token utility for `asyncio`-based code.
|
|
184
|
+
|
|
185
|
+
### The underlying Python pain points
|
|
186
|
+
|
|
187
|
+
- **[Graceful exit from ThreadPoolExecutor when blocked on IO — discuss.python.org](https://discuss.python.org/t/graceful-exit-from-threadpoolexecutor-when-blocked-on-io-problem-and-possible-enhancement/80380)**
|
|
188
|
+
Ongoing discussion acknowledging that Python has no clean way to cancel a worker that is blocked on I/O. This module is effectively a targeted workaround for the HTTP-specific case.
|
|
189
|
+
|
|
190
|
+
- **[`threading` — Python docs](https://docs.python.org/3/library/threading.html)**
|
|
191
|
+
`Thread` has no `cancel()` / `interrupt()`; cooperation via `Event` is the only sanctioned approach.
|
|
192
|
+
|
|
193
|
+
- **[`Session.close()` does not close underlying sockets — psf/requests#5633](https://github.com/psf/requests/issues/5633)**
|
|
194
|
+
Illustrates why "just use `requests` and close the session" is not a reliable answer.
|
|
195
|
+
|
|
196
|
+
- **[Unclosed socket in urllib when ftp request times out after connect — cpython#140691](https://github.com/python/cpython/issues/140691)**
|
|
197
|
+
A related stdlib lifecycle bug — background for why we deliberately take full control of the connection object.
|
|
198
|
+
|
|
199
|
+
### Related stdlib primitives this module builds on
|
|
200
|
+
|
|
201
|
+
- **[`http.client` — Python docs](https://docs.python.org/3/library/http.client.html)**
|
|
202
|
+
the low-level HTTP protocol implementation we wrap.
|
|
203
|
+
- **[`threading.Event` — Python docs](https://docs.python.org/3/library/threading.html#event-objects)**
|
|
204
|
+
used internally to signal completion.
|
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
# cancellable_http_client
|
|
2
|
+
|
|
3
|
+
A tiny, dependency-free HTTP client for Python with **cancellable in-flight requests** and **hard wall-clock timeout**.
|
|
4
|
+
|
|
5
|
+
- Standard library only — no `requests`, no `httpx`, no `urllib3`.
|
|
6
|
+
- Single file, ~300 lines.
|
|
7
|
+
- Synchronous API that plays well with `threading`-based workers.
|
|
8
|
+
- Safe `close()` from any thread, at any time, including mid-transfer.
|
|
9
|
+
- Hard wall-clock `timeout` that bounds the entire request.
|
|
10
|
+
|
|
11
|
+
## Why it exists
|
|
12
|
+
|
|
13
|
+
Python has no clean way to interrupt a thread that is blocked on a socket read. `concurrent.futures.Future.cancel()` does nothing once the task has started, and `requests` / `urllib.urlopen()` give you no handle to abort an in-flight request.
|
|
14
|
+
|
|
15
|
+
The one primitive that *does* work is closing the underlying socket: any pending `recv()` immediately unblocks with an error. This module wraps that trick behind a tiny, boring API so you don't have to reinvent it — or worry about the lifecycle edge cases — every time you need it.
|
|
16
|
+
|
|
17
|
+
If your codebase is built around `asyncio`, you don't need this; use `httpx` and `task.cancel()` instead. This module targets the very real case where you have existing threaded code and you want one HTTP call in the middle of it to be cancellable, without rewriting everything to be async.
|
|
18
|
+
|
|
19
|
+
## Usage
|
|
20
|
+
|
|
21
|
+
```python
|
|
22
|
+
import time
|
|
23
|
+
import cancellable_http_client as client
|
|
24
|
+
|
|
25
|
+
req = client.Request("https://example.com/")
|
|
26
|
+
req.start() # the actual TCP connection happens here
|
|
27
|
+
start = time.monotonic()
|
|
28
|
+
while not req.done:
|
|
29
|
+
if time.monotonic() - start > 5:
|
|
30
|
+
print("taking too long, aborting...")
|
|
31
|
+
req.close() # interrupts the request if it's still in-flight
|
|
32
|
+
req.wait(0.1) # wait a bit before checking again
|
|
33
|
+
if req.error:
|
|
34
|
+
print(f"failed: {req.error}")
|
|
35
|
+
elif req.response and req.response.status == 200:
|
|
36
|
+
print(req.response.body)
|
|
37
|
+
req.close() # safe to call any time, even mid-flight
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
You can also use it as a context manager:
|
|
41
|
+
|
|
42
|
+
```python
|
|
43
|
+
with client.Request("https://example.com/") as req:
|
|
44
|
+
req.start()
|
|
45
|
+
req.wait(timeout=5)
|
|
46
|
+
...
|
|
47
|
+
# close() is called automatically on exit
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### Sharing a thread pool
|
|
51
|
+
|
|
52
|
+
By default each `Request` spawns its own daemon thread. To reuse a pool instead, assign an `Executor` to the module-level attribute:
|
|
53
|
+
|
|
54
|
+
```python
|
|
55
|
+
from concurrent.futures import ThreadPoolExecutor
|
|
56
|
+
client.executor = ThreadPoolExecutor(max_workers=8)
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## API
|
|
60
|
+
|
|
61
|
+
### `Request(url, method="GET", headers=None, body=b"", socket_timeout=30, timeout=None)`
|
|
62
|
+
|
|
63
|
+
Construct a request. No network I/O happens here — connection failures are reported via `error` after `start()`.
|
|
64
|
+
|
|
65
|
+
- **`socket_timeout`** — per-socket-operation timeout in seconds, passed to `http.client.HTTPConnection`.
|
|
66
|
+
- **`timeout`** — wall-clock limit in seconds for the entire request. Triggers `close()` automatically if the request is not done in time. `None` disables.
|
|
67
|
+
- **`start()`** — kick off the request. Non-blocking.
|
|
68
|
+
- **`wait(timeout=None) -> bool`** — block until the request finishes. Returns `True` on completion, `False` on timeout.
|
|
69
|
+
- **`close()`** — abort the request and release resources. Safe to call any time, from any thread, any number of times.
|
|
70
|
+
- **`done`** *(property)* — `True` once the request has finished (success, failure, or close).
|
|
71
|
+
- **`response`** — a `Response` object on success, otherwise `None`.
|
|
72
|
+
- **`error`** — the exception raised during the request, or `None`.
|
|
73
|
+
|
|
74
|
+
### `Response`
|
|
75
|
+
|
|
76
|
+
A read-only, socket-free container exposing the same attributes as `http.client.HTTPResponse`:
|
|
77
|
+
|
|
78
|
+
- `status`, `reason`, `version`
|
|
79
|
+
- `headers` (an `http.client.HTTPMessage`)
|
|
80
|
+
- `body` (`bytes`, eagerly read)
|
|
81
|
+
- `getheader(name, default=None)`, `getheaders()`
|
|
82
|
+
|
|
83
|
+
## Robust timeout
|
|
84
|
+
|
|
85
|
+
Most Python HTTP clients set a *per-socket-operation* timeout (`socket.settimeout`). This leaves several gaps:
|
|
86
|
+
|
|
87
|
+
- **Slow drip** — a server that sends one byte every 29 seconds never triggers a 30-second socket timeout, yet the total transfer can take arbitrarily long.
|
|
88
|
+
- **DNS resolution** — `socket.getaddrinfo()` is a blocking C library call with no timeout parameter. Python cannot interrupt it.
|
|
89
|
+
- **Total elapsed time** — there is no built-in way to cap the wall-clock time of an entire request across connection, TLS handshake, sending, and receiving.
|
|
90
|
+
|
|
91
|
+
`cancellable_http_client` addresses this with two separate knobs:
|
|
92
|
+
|
|
93
|
+
| Parameter | Scope | Default |
|
|
94
|
+
|---|---|---|
|
|
95
|
+
| `socket_timeout` | Per socket operation (connect, send, recv) | 30 s |
|
|
96
|
+
| `timeout` | Wall-clock limit on the entire request | None (no limit) |
|
|
97
|
+
|
|
98
|
+
When `timeout` fires it calls `close()`, which immediately unblocks any pending socket operation by closing the underlying connection. This gives you a hard upper bound on how long `wait()` will block — something that `socket_timeout` alone cannot guarantee.
|
|
99
|
+
|
|
100
|
+
## Comparison with existing libraries
|
|
101
|
+
|
|
102
|
+
| | cancellable_http_client | httpx | requests |
|
|
103
|
+
|---|---|---|---|
|
|
104
|
+
| Cancel an in-flight request from another thread | ✅ | ⚠️ async, [unreliable](https://github.com/encode/httpx/issues/1461) | ⚠️ hacky |
|
|
105
|
+
| Hard wall-clock timeout on entire request | ✅ | ⚠️ per-operation | ⚠️ per-operation |
|
|
106
|
+
| Synchronous API | ✅ | ✅ (also async) | ✅ |
|
|
107
|
+
| No third-party dependencies | ✅ | ❌ | ❌ |
|
|
108
|
+
| Line count | ~300 | thousands | thousands |
|
|
109
|
+
| Fits a threading-based worker | ✅ | ❌ | ⚠️ |
|
|
110
|
+
| Redirects, cookies, User-Agent | ⚠️ manual | ✅ | ✅ |
|
|
111
|
+
|
|
112
|
+
`httpx` is a good choice if you are already in an `asyncio` world, though `task.cancel()` on in-flight requests [can leave the connection pool in a broken state](https://github.com/encode/httpx/issues/1461). `requests` does not offer a reliable way to interrupt an in-flight call; `Session.close()` does not forcibly close active sockets ([psf/requests#5633](https://github.com/psf/requests/issues/5633)).
|
|
113
|
+
|
|
114
|
+
## Limitations
|
|
115
|
+
|
|
116
|
+
This library is a thin wrapper around `http.client` and does not provide the high-level conveniences found in `requests` or `httpx`:
|
|
117
|
+
|
|
118
|
+
- No automatic redirect following
|
|
119
|
+
- No cookie management
|
|
120
|
+
- No default User-Agent header
|
|
121
|
+
- No HTTP/2 support
|
|
122
|
+
|
|
123
|
+
These are all `http.client` limitations, not restrictions added by this library. You can still handle them manually via the `headers` parameter.
|
|
124
|
+
|
|
125
|
+
## Tests
|
|
126
|
+
|
|
127
|
+
```
|
|
128
|
+
python -m unittest discover -s tests -v
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
No third-party test dependencies. Tests use local throwaway servers (normal, slow, blackhole, mid-body disconnect) to exercise cancellation, timeout, and error paths without touching the network.
|
|
132
|
+
|
|
133
|
+
## License
|
|
134
|
+
|
|
135
|
+
Copyright 2026 Sakilabo Corporation Ltd.
|
|
136
|
+
Licensed under the Universal Permissive License v 1.0
|
|
137
|
+
([UPL-1.0](https://oss.oracle.com/licenses/upl/)).
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## References (for the eventual public release)
|
|
142
|
+
|
|
143
|
+
Background reading and related work collected during the design of this module. Useful as citations in the README or a launch blog post.
|
|
144
|
+
|
|
145
|
+
### Prior art / closest existing work
|
|
146
|
+
|
|
147
|
+
- **["TIL: Stopping Requests Mid Flight" — haykot.dev](https://haykot.dev/blog/til-stopping-requests-mid-flight/)**
|
|
148
|
+
A blog post describing the "close the socket from another thread" trick as a personal discovery. The same underlying idea as this module, but kept as a snippet rather than packaged as a library.
|
|
149
|
+
|
|
150
|
+
- **[httpcore on PyPI](https://pypi.org/project/httpcore/)**
|
|
151
|
+
the low-level HTTP engine behind `httpx`. Supports cancellation via async task cancellation; relies on `asyncio` or `trio`.
|
|
152
|
+
|
|
153
|
+
- **[HTTPX](https://www.python-httpx.org/)**
|
|
154
|
+
modern high-level HTTP client; cancellation is done via `task.cancel()` in an async context.
|
|
155
|
+
|
|
156
|
+
- **[asyncio-cancel-token](https://asyncio-cancel-token.readthedocs.io/en/latest/cancel_token.html)**
|
|
157
|
+
a cancellation-token utility for `asyncio`-based code.
|
|
158
|
+
|
|
159
|
+
### The underlying Python pain points
|
|
160
|
+
|
|
161
|
+
- **[Graceful exit from ThreadPoolExecutor when blocked on IO — discuss.python.org](https://discuss.python.org/t/graceful-exit-from-threadpoolexecutor-when-blocked-on-io-problem-and-possible-enhancement/80380)**
|
|
162
|
+
Ongoing discussion acknowledging that Python has no clean way to cancel a worker that is blocked on I/O. This module is effectively a targeted workaround for the HTTP-specific case.
|
|
163
|
+
|
|
164
|
+
- **[`threading` — Python docs](https://docs.python.org/3/library/threading.html)**
|
|
165
|
+
`Thread` has no `cancel()` / `interrupt()`; cooperation via `Event` is the only sanctioned approach.
|
|
166
|
+
|
|
167
|
+
- **[`Session.close()` does not close underlying sockets — psf/requests#5633](https://github.com/psf/requests/issues/5633)**
|
|
168
|
+
Illustrates why "just use `requests` and close the session" is not a reliable answer.
|
|
169
|
+
|
|
170
|
+
- **[Unclosed socket in urllib when ftp request times out after connect — cpython#140691](https://github.com/python/cpython/issues/140691)**
|
|
171
|
+
A related stdlib lifecycle bug — background for why we deliberately take full control of the connection object.
|
|
172
|
+
|
|
173
|
+
### Related stdlib primitives this module builds on
|
|
174
|
+
|
|
175
|
+
- **[`http.client` — Python docs](https://docs.python.org/3/library/http.client.html)**
|
|
176
|
+
the low-level HTTP protocol implementation we wrap.
|
|
177
|
+
- **[`threading.Event` — Python docs](https://docs.python.org/3/library/threading.html#event-objects)**
|
|
178
|
+
used internally to signal completion.
|
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: cancellable-http-client
|
|
3
|
+
Version: 1.0
|
|
4
|
+
Summary: A tiny, dependency-free HTTP client for Python with cancellable in-flight requests and hard wall-clock timeout.
|
|
5
|
+
Author: Sakilabo Corporation Ltd.
|
|
6
|
+
License-Expression: UPL-1.0
|
|
7
|
+
Project-URL: Homepage, https://github.com/sakilabo/cancellable-http-client
|
|
8
|
+
Project-URL: Issues, https://github.com/sakilabo/cancellable-http-client/issues
|
|
9
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.7
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
21
|
+
Classifier: Topic :: Internet :: WWW/HTTP
|
|
22
|
+
Requires-Python: >=3.7
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
License-File: LICENSE
|
|
25
|
+
Dynamic: license-file
|
|
26
|
+
|
|
27
|
+
# cancellable_http_client
|
|
28
|
+
|
|
29
|
+
A tiny, dependency-free HTTP client for Python with **cancellable in-flight requests** and **hard wall-clock timeout**.
|
|
30
|
+
|
|
31
|
+
- Standard library only — no `requests`, no `httpx`, no `urllib3`.
|
|
32
|
+
- Single file, ~300 lines.
|
|
33
|
+
- Synchronous API that plays well with `threading`-based workers.
|
|
34
|
+
- Safe `close()` from any thread, at any time, including mid-transfer.
|
|
35
|
+
- Hard wall-clock `timeout` that bounds the entire request.
|
|
36
|
+
|
|
37
|
+
## Why it exists
|
|
38
|
+
|
|
39
|
+
Python has no clean way to interrupt a thread that is blocked on a socket read. `concurrent.futures.Future.cancel()` does nothing once the task has started, and `requests` / `urllib.urlopen()` give you no handle to abort an in-flight request.
|
|
40
|
+
|
|
41
|
+
The one primitive that *does* work is closing the underlying socket: any pending `recv()` immediately unblocks with an error. This module wraps that trick behind a tiny, boring API so you don't have to reinvent it — or worry about the lifecycle edge cases — every time you need it.
|
|
42
|
+
|
|
43
|
+
If your codebase is built around `asyncio`, you don't need this; use `httpx` and `task.cancel()` instead. This module targets the very real case where you have existing threaded code and you want one HTTP call in the middle of it to be cancellable, without rewriting everything to be async.
|
|
44
|
+
|
|
45
|
+
## Usage
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
import time
|
|
49
|
+
import cancellable_http_client as client
|
|
50
|
+
|
|
51
|
+
req = client.Request("https://example.com/")
|
|
52
|
+
req.start() # the actual TCP connection happens here
|
|
53
|
+
start = time.monotonic()
|
|
54
|
+
while not req.done:
|
|
55
|
+
if time.monotonic() - start > 5:
|
|
56
|
+
print("taking too long, aborting...")
|
|
57
|
+
req.close() # interrupts the request if it's still in-flight
|
|
58
|
+
req.wait(0.1) # wait a bit before checking again
|
|
59
|
+
if req.error:
|
|
60
|
+
print(f"failed: {req.error}")
|
|
61
|
+
elif req.response and req.response.status == 200:
|
|
62
|
+
print(req.response.body)
|
|
63
|
+
req.close() # safe to call any time, even mid-flight
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
You can also use it as a context manager:
|
|
67
|
+
|
|
68
|
+
```python
|
|
69
|
+
with client.Request("https://example.com/") as req:
|
|
70
|
+
req.start()
|
|
71
|
+
req.wait(timeout=5)
|
|
72
|
+
...
|
|
73
|
+
# close() is called automatically on exit
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Sharing a thread pool
|
|
77
|
+
|
|
78
|
+
By default each `Request` spawns its own daemon thread. To reuse a pool instead, assign an `Executor` to the module-level attribute:
|
|
79
|
+
|
|
80
|
+
```python
|
|
81
|
+
from concurrent.futures import ThreadPoolExecutor
|
|
82
|
+
client.executor = ThreadPoolExecutor(max_workers=8)
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
## API
|
|
86
|
+
|
|
87
|
+
### `Request(url, method="GET", headers=None, body=b"", socket_timeout=30, timeout=None)`
|
|
88
|
+
|
|
89
|
+
Construct a request. No network I/O happens here — connection failures are reported via `error` after `start()`.
|
|
90
|
+
|
|
91
|
+
- **`socket_timeout`** — per-socket-operation timeout in seconds, passed to `http.client.HTTPConnection`.
|
|
92
|
+
- **`timeout`** — wall-clock limit in seconds for the entire request. Triggers `close()` automatically if the request is not done in time. `None` disables.
|
|
93
|
+
- **`start()`** — kick off the request. Non-blocking.
|
|
94
|
+
- **`wait(timeout=None) -> bool`** — block until the request finishes. Returns `True` on completion, `False` on timeout.
|
|
95
|
+
- **`close()`** — abort the request and release resources. Safe to call any time, from any thread, any number of times.
|
|
96
|
+
- **`done`** *(property)* — `True` once the request has finished (success, failure, or close).
|
|
97
|
+
- **`response`** — a `Response` object on success, otherwise `None`.
|
|
98
|
+
- **`error`** — the exception raised during the request, or `None`.
|
|
99
|
+
|
|
100
|
+
### `Response`
|
|
101
|
+
|
|
102
|
+
A read-only, socket-free container exposing the same attributes as `http.client.HTTPResponse`:
|
|
103
|
+
|
|
104
|
+
- `status`, `reason`, `version`
|
|
105
|
+
- `headers` (an `http.client.HTTPMessage`)
|
|
106
|
+
- `body` (`bytes`, eagerly read)
|
|
107
|
+
- `getheader(name, default=None)`, `getheaders()`
|
|
108
|
+
|
|
109
|
+
## Robust timeout
|
|
110
|
+
|
|
111
|
+
Most Python HTTP clients set a *per-socket-operation* timeout (`socket.settimeout`). This leaves several gaps:
|
|
112
|
+
|
|
113
|
+
- **Slow drip** — a server that sends one byte every 29 seconds never triggers a 30-second socket timeout, yet the total transfer can take arbitrarily long.
|
|
114
|
+
- **DNS resolution** — `socket.getaddrinfo()` is a blocking C library call with no timeout parameter. Python cannot interrupt it.
|
|
115
|
+
- **Total elapsed time** — there is no built-in way to cap the wall-clock time of an entire request across connection, TLS handshake, sending, and receiving.
|
|
116
|
+
|
|
117
|
+
`cancellable_http_client` addresses this with two separate knobs:
|
|
118
|
+
|
|
119
|
+
| Parameter | Scope | Default |
|
|
120
|
+
|---|---|---|
|
|
121
|
+
| `socket_timeout` | Per socket operation (connect, send, recv) | 30 s |
|
|
122
|
+
| `timeout` | Wall-clock limit on the entire request | None (no limit) |
|
|
123
|
+
|
|
124
|
+
When `timeout` fires it calls `close()`, which immediately unblocks any pending socket operation by closing the underlying connection. This gives you a hard upper bound on how long `wait()` will block — something that `socket_timeout` alone cannot guarantee.
|
|
125
|
+
|
|
126
|
+
## Comparison with existing libraries
|
|
127
|
+
|
|
128
|
+
| | cancellable_http_client | httpx | requests |
|
|
129
|
+
|---|---|---|---|
|
|
130
|
+
| Cancel an in-flight request from another thread | ✅ | ⚠️ async, [unreliable](https://github.com/encode/httpx/issues/1461) | ⚠️ hacky |
|
|
131
|
+
| Hard wall-clock timeout on entire request | ✅ | ⚠️ per-operation | ⚠️ per-operation |
|
|
132
|
+
| Synchronous API | ✅ | ✅ (also async) | ✅ |
|
|
133
|
+
| No third-party dependencies | ✅ | ❌ | ❌ |
|
|
134
|
+
| Line count | ~300 | thousands | thousands |
|
|
135
|
+
| Fits a threading-based worker | ✅ | ❌ | ⚠️ |
|
|
136
|
+
| Redirects, cookies, User-Agent | ⚠️ manual | ✅ | ✅ |
|
|
137
|
+
|
|
138
|
+
`httpx` is a good choice if you are already in an `asyncio` world, though `task.cancel()` on in-flight requests [can leave the connection pool in a broken state](https://github.com/encode/httpx/issues/1461). `requests` does not offer a reliable way to interrupt an in-flight call; `Session.close()` does not forcibly close active sockets ([psf/requests#5633](https://github.com/psf/requests/issues/5633)).
|
|
139
|
+
|
|
140
|
+
## Limitations
|
|
141
|
+
|
|
142
|
+
This library is a thin wrapper around `http.client` and does not provide the high-level conveniences found in `requests` or `httpx`:
|
|
143
|
+
|
|
144
|
+
- No automatic redirect following
|
|
145
|
+
- No cookie management
|
|
146
|
+
- No default User-Agent header
|
|
147
|
+
- No HTTP/2 support
|
|
148
|
+
|
|
149
|
+
These are all `http.client` limitations, not restrictions added by this library. You can still handle them manually via the `headers` parameter.
|
|
150
|
+
|
|
151
|
+
## Tests
|
|
152
|
+
|
|
153
|
+
```
|
|
154
|
+
python -m unittest discover -s tests -v
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
No third-party test dependencies. Tests use local throwaway servers (normal, slow, blackhole, mid-body disconnect) to exercise cancellation, timeout, and error paths without touching the network.
|
|
158
|
+
|
|
159
|
+
## License
|
|
160
|
+
|
|
161
|
+
Copyright 2026 Sakilabo Corporation Ltd.
|
|
162
|
+
Licensed under the Universal Permissive License v 1.0
|
|
163
|
+
([UPL-1.0](https://oss.oracle.com/licenses/upl/)).
|
|
164
|
+
|
|
165
|
+
---
|
|
166
|
+
|
|
167
|
+
## References (for the eventual public release)
|
|
168
|
+
|
|
169
|
+
Background reading and related work collected during the design of this module. Useful as citations in the README or a launch blog post.
|
|
170
|
+
|
|
171
|
+
### Prior art / closest existing work
|
|
172
|
+
|
|
173
|
+
- **["TIL: Stopping Requests Mid Flight" — haykot.dev](https://haykot.dev/blog/til-stopping-requests-mid-flight/)**
|
|
174
|
+
A blog post describing the "close the socket from another thread" trick as a personal discovery. The same underlying idea as this module, but kept as a snippet rather than packaged as a library.
|
|
175
|
+
|
|
176
|
+
- **[httpcore on PyPI](https://pypi.org/project/httpcore/)**
|
|
177
|
+
the low-level HTTP engine behind `httpx`. Supports cancellation via async task cancellation; relies on `asyncio` or `trio`.
|
|
178
|
+
|
|
179
|
+
- **[HTTPX](https://www.python-httpx.org/)**
|
|
180
|
+
modern high-level HTTP client; cancellation is done via `task.cancel()` in an async context.
|
|
181
|
+
|
|
182
|
+
- **[asyncio-cancel-token](https://asyncio-cancel-token.readthedocs.io/en/latest/cancel_token.html)**
|
|
183
|
+
a cancellation-token utility for `asyncio`-based code.
|
|
184
|
+
|
|
185
|
+
### The underlying Python pain points
|
|
186
|
+
|
|
187
|
+
- **[Graceful exit from ThreadPoolExecutor when blocked on IO — discuss.python.org](https://discuss.python.org/t/graceful-exit-from-threadpoolexecutor-when-blocked-on-io-problem-and-possible-enhancement/80380)**
|
|
188
|
+
Ongoing discussion acknowledging that Python has no clean way to cancel a worker that is blocked on I/O. This module is effectively a targeted workaround for the HTTP-specific case.
|
|
189
|
+
|
|
190
|
+
- **[`threading` — Python docs](https://docs.python.org/3/library/threading.html)**
|
|
191
|
+
`Thread` has no `cancel()` / `interrupt()`; cooperation via `Event` is the only sanctioned approach.
|
|
192
|
+
|
|
193
|
+
- **[`Session.close()` does not close underlying sockets — psf/requests#5633](https://github.com/psf/requests/issues/5633)**
|
|
194
|
+
Illustrates why "just use `requests` and close the session" is not a reliable answer.
|
|
195
|
+
|
|
196
|
+
- **[Unclosed socket in urllib when ftp request times out after connect — cpython#140691](https://github.com/python/cpython/issues/140691)**
|
|
197
|
+
A related stdlib lifecycle bug — background for why we deliberately take full control of the connection object.
|
|
198
|
+
|
|
199
|
+
### Related stdlib primitives this module builds on
|
|
200
|
+
|
|
201
|
+
- **[`http.client` — Python docs](https://docs.python.org/3/library/http.client.html)**
|
|
202
|
+
the low-level HTTP protocol implementation we wrap.
|
|
203
|
+
- **[`threading.Event` — Python docs](https://docs.python.org/3/library/threading.html#event-objects)**
|
|
204
|
+
used internally to signal completion.
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
cancellable_http_client.py
|
|
4
|
+
pyproject.toml
|
|
5
|
+
cancellable_http_client.egg-info/PKG-INFO
|
|
6
|
+
cancellable_http_client.egg-info/SOURCES.txt
|
|
7
|
+
cancellable_http_client.egg-info/dependency_links.txt
|
|
8
|
+
cancellable_http_client.egg-info/top_level.txt
|
|
9
|
+
tests/test_cancellable_http_client.py
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
cancellable_http_client
|