httpperf 1.0rc4__tar.gz → 1.0rc6__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {httpperf-1.0rc4/httpperf.egg-info → httpperf-1.0rc6}/PKG-INFO +134 -7
- {httpperf-1.0rc4 → httpperf-1.0rc6}/README.md +133 -6
- {httpperf-1.0rc4 → httpperf-1.0rc6}/httpperf/__init__.py +243 -28
- httpperf-1.0rc6/httpperf/__main__.py +5 -0
- httpperf-1.0rc6/httpperf/_cli.py +808 -0
- httpperf-1.0rc6/httpperf/version.py +1 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6/httpperf.egg-info}/PKG-INFO +134 -7
- {httpperf-1.0rc4 → httpperf-1.0rc6}/httpperf.egg-info/SOURCES.txt +11 -1
- httpperf-1.0rc6/httpperf.egg-info/entry_points.txt +2 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/pyproject.toml +6 -3
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/httpperf.c +1146 -96
- httpperf-1.0rc6/tests/test_api.py +695 -0
- httpperf-1.0rc6/tests/test_cancel.py +89 -0
- httpperf-1.0rc6/tests/test_cli.py +175 -0
- httpperf-1.0rc6/tests/test_connections.py +328 -0
- httpperf-1.0rc6/tests/test_cookie.py +15 -0
- httpperf-1.0rc6/tests/test_discard_body.py +33 -0
- httpperf-1.0rc6/tests/test_engine.py +272 -0
- httpperf-1.0rc6/tests/test_host_stats.py +41 -0
- httpperf-1.0rc6/tests/test_http_versions.py +178 -0
- httpperf-1.0rc6/tests/test_httpbin.py +325 -0
- httpperf-1.0rc6/tests/test_leaks.py +50 -0
- httpperf-1.0rc6/tests/test_limiter.py +207 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/tests/test_response.py +37 -9
- httpperf-1.0rc6/tests/test_stats.py +102 -0
- httpperf-1.0rc6/tests/test_stream.py +97 -0
- httpperf-1.0rc6/tests/test_threads.py +83 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/tests/test_tls.py +102 -71
- httpperf-1.0rc6/tests/test_upload.py +161 -0
- httpperf-1.0rc4/httpperf/version.py +0 -1
- httpperf-1.0rc4/tests/test_api.py +0 -542
- httpperf-1.0rc4/tests/test_cancel.py +0 -138
- httpperf-1.0rc4/tests/test_connections.py +0 -304
- httpperf-1.0rc4/tests/test_cookie.py +0 -14
- httpperf-1.0rc4/tests/test_engine.py +0 -328
- httpperf-1.0rc4/tests/test_http_versions.py +0 -201
- httpperf-1.0rc4/tests/test_httpbin.py +0 -289
- httpperf-1.0rc4/tests/test_leaks.py +0 -41
- httpperf-1.0rc4/tests/test_limiter.py +0 -239
- {httpperf-1.0rc4 → httpperf-1.0rc6}/LICENSE +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/MANIFEST.in +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/NOTICE +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/httpperf.egg-info/dependency_links.txt +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/httpperf.egg-info/top_level.txt +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/setup.cfg +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/setup.py +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/.clang-format +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/ae.c +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/ae.h +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/ae_epoll.c +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/ae_evport.c +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/ae_kqueue.c +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/ae_select.c +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/atomicvar.h +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/config.h +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/zmalloc.c +0 -0
- {httpperf-1.0rc4 → httpperf-1.0rc6}/src/ae/zmalloc.h +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: httpperf
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.0rc6
|
|
4
4
|
Summary: An asyncio HTTP client for load testing, built on libcurl.
|
|
5
5
|
Author-email: Tony Simpson <agjasimpson@gmail.com>
|
|
6
6
|
Project-URL: Source, https://github.com/tonysimpson/httpperf
|
|
@@ -50,7 +50,7 @@ Bytes are read off the wire as they arrive, whatever Python is doing. If Python
|
|
|
50
50
|
pip install httpperf
|
|
51
51
|
```
|
|
52
52
|
|
|
53
|
-
Wheels are published for Linux on x86_64 and aarch64, for Python 3.9 to 3.15, with glibc 2.34 or later (Ubuntu 22.04, Debian 12, RHEL 9 and later) or musl (Alpine). They include their own libcurl, with HTTP/2 and HTTP/3.
|
|
53
|
+
Wheels are published for Linux on x86_64 and aarch64, for Python 3.9 to 3.15 and free-threaded 3.14 and 3.15 (`python3.14t`, `python3.15t`), with glibc 2.34 or later (Ubuntu 22.04, Debian 12, RHEL 9 and later) or musl (Alpine). They include their own libcurl, with HTTP/2 and HTTP/3.
|
|
54
54
|
|
|
55
55
|
Anywhere else, including Python 3.8, pip builds from source against the system's libcurl, which needs its development headers and a C compiler. That libcurl decides whether HTTP/3 is available: `curl -V` lists `HTTP3` under Features when it does.
|
|
56
56
|
|
|
@@ -86,6 +86,22 @@ asyncio.run(main())
|
|
|
86
86
|
|
|
87
87
|
Sessions are cheap: create one per virtual user or journey. They share a default `Engine` (the curl thread, the shared connection pool and the TLS cache) unless you pass your own.
|
|
88
88
|
|
|
89
|
+
## Command line
|
|
90
|
+
|
|
91
|
+
`python -m httpperf` is a load generator in the spirit of wrk and wrk2. Find the most a server can handle by trying more requests in flight:
|
|
92
|
+
|
|
93
|
+
```
|
|
94
|
+
python -m httpperf -c 10,50,200,1000 http://localhost:8080/
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Then measure latency at fixed rates below that peak:
|
|
98
|
+
|
|
99
|
+
```
|
|
100
|
+
python -m httpperf --rate 20000,40000,60000 http://localhost:8080/
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Each step reports requests per second, latency percentiles, errors and transfer rates, and warns when the client rather than the server was the limit. A short Python script can define the request to make. See [docs/cli.md](docs/cli.md) for all the options.
|
|
104
|
+
|
|
89
105
|
## API
|
|
90
106
|
|
|
91
107
|
### `Session(*, engine=None, ...)`
|
|
@@ -105,7 +121,7 @@ All arguments are keyword-only. A session can be created anywhere, even outside
|
|
|
105
121
|
| `cert` | `None` | `(certfile, keyfile)` paths for a client certificate, used by every request unless the request overrides it. |
|
|
106
122
|
| `timeout` | `None` | Seconds allowed for each request, from the start of the transfer to the last byte (each redirect hop separately). `None` or 0 means no limit. Cancelling the task (or `asyncio.wait_for`) also stops the transfer and frees its connection slot; on HTTP/1.1 that closes the connection, on HTTP/2 and HTTP/3 it resets only that stream. |
|
|
107
123
|
| `connect_timeout` | `None` | Seconds allowed to connect, TLS included. `None` means curl's default of 300 s. |
|
|
108
|
-
| `verify` | `
|
|
124
|
+
| `verify` | `True` | Server certificate checking. `False`: none, for test servers with self-signed certificates. `True`: check against the system CAs, from the first of: `SSL_CERT_FILE` and `SSL_CERT_DIR`, if set and present; libcurl's own default bundle, if present; Python's default bundle (`ssl.get_default_verify_paths()`); the usual system locations. A path: check against that CA bundle. Requests can override it. |
|
|
109
125
|
| `json_dumps` | `None` | Function that turns a `json=` argument into the request body (`str` or `bytes`). `None` means the engine's. |
|
|
110
126
|
| `json_loads` | `None` | Function that turns the response body (`bytes`) into the result of `json()`. `None` means the engine's. |
|
|
111
127
|
|
|
@@ -113,7 +129,7 @@ Proxies come from the environment, as curl reads them: `http_proxy` (not `HTTP_P
|
|
|
113
129
|
|
|
114
130
|
Cookies always belong to the session, whatever the connection settings.
|
|
115
131
|
|
|
116
|
-
### `Engine(*, max_host_connections=0, max_total_connections=0, max_idle_connections=1000, json_dumps=..., json_loads=...)`
|
|
132
|
+
### `Engine(*, max_host_connections=0, max_total_connections=0, max_idle_connections=1000, buffer_size=16384, host_stats=False, json_dumps=..., json_loads=...)`
|
|
117
133
|
|
|
118
134
|
What sessions share: the curl thread, the shared connection pool, the TLS session cache and the limits. Most programs never create one. Create one to set limits, or to keep groups of sessions in separate pools, and pass it to `Session(engine=...)`.
|
|
119
135
|
|
|
@@ -122,14 +138,29 @@ What sessions share: the curl thread, the shared connection pool, the TLS sessio
|
|
|
122
138
|
| `max_host_connections` | Cap on connections per host for the engine's sessions that use the shared pool or no pool. 0 means no limit. httpperf queues requests above it and reports the wait as `queue_time`. |
|
|
123
139
|
| `max_total_connections` | Cap on all connections for the same sessions. 0 means no limit. |
|
|
124
140
|
| `max_idle_connections` | How many connections each pool keeps (the shared pool, and each session's own pool). Must be at least 1. |
|
|
141
|
+
| `buffer_size` | Bytes curl reads from a connection at a time, from 1024 to 10 MB. The default, curl's own, suits many small responses. For large downloads 256 KB is much faster (here 14 to 23 GB/s with `discard_body`, 10 to 15 GB/s streamed), but each transfer in flight has its own buffer, so it costs memory at high connection counts. |
|
|
142
|
+
| `host_stats` | `True` keeps counts for each host, read with `engine.host_stats()`. See [Statistics](#statistics). |
|
|
125
143
|
| `json_dumps`, `json_loads` | The JSON functions for the engine's sessions, unless a session sets its own. The defaults use the standard library: compact output, and bodies decoded as UTF-8. See [JSON](#json). |
|
|
126
144
|
|
|
127
145
|
Each asyncio loop has its own default engine, so sessions without an engine work in any thread, and in each `asyncio.run()`, without setup. A loop's default engine lasts as long as the loop, and is closed once the loop has been closed and no session uses it.
|
|
128
146
|
|
|
129
147
|
An engine binds to the asyncio loop running when it is first used, and so does a session. An engine serves one loop at a time, and using it from a second live loop raises `RuntimeError`. Once its loop is closed, as at the end of `asyncio.run()`, it binds to the next one and keeps its pool. So a session kept between `asyncio.run()` calls keeps its connections. To run loops in several threads at once with your own engines, give each thread its own `Engine`.
|
|
130
148
|
|
|
149
|
+
On free-threaded Python (3.14t and later), httpperf runs without the GIL, so threads that each run their own loop and engine run their Python code in parallel. That is what helps when Python work on each response is the limit, as it usually is in a real load test. On 4 cores, with 20 µs of Python work per response, 4 threads went from 42k requests per second with the GIL to 111k without it; with 100 µs of work, from 9.6k to 34k. With little Python work per response the GIL costs little, since the curl threads never hold it.
|
|
150
|
+
|
|
131
151
|
`engine.close()` stops the curl thread and closes the shared pool's connections, and later requests raise `RuntimeError`. Requests still in flight fail with `RequestError`, and a `ResourceWarning` says how many there were. `with` and `async with` call `close()` on exit. An engine that is no longer referenced is closed automatically. See [docs/connections.md](docs/connections.md) for how these fit together.
|
|
132
152
|
|
|
153
|
+
`engine.stats()` gives the counts for all the engine's sessions. See [Statistics](#statistics).
|
|
154
|
+
|
|
155
|
+
Each engine has one curl thread. When the Python code does real work on each response, Python is the limit and one engine is plenty. A simple load driver that does little per response can make the curl thread the limit instead. Several engines can share one asyncio loop, so spreading sessions over them drives more throughput:
|
|
156
|
+
|
|
157
|
+
```python
|
|
158
|
+
engines = [httpperf.Engine() for _ in range(4)]
|
|
159
|
+
sessions = [httpperf.Session(engine=engines[i % len(engines)]) for i in range(100)]
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Each engine has its own connections and limits. On 4 cores against a local server, 4 engines took a closed loop of 100 requests from 104k to 223k requests per second for small responses, and from 60k to 183k for 100 KB ones, at which point the Python thread was the limit.
|
|
163
|
+
|
|
133
164
|
### Session methods
|
|
134
165
|
|
|
135
166
|
- `await session.get(url, **kwargs)`, and likewise `post`, `put`, `patch`, `delete`, `head` and `options`.
|
|
@@ -138,6 +169,8 @@ An engine binds to the asyncio loop running when it is first used, and so does a
|
|
|
138
169
|
- `session.cookies()`: the cookie jar as a dict.
|
|
139
170
|
- `session.get_cookie_list()`, `session.add_cookie_list(cookies)`, `session.erase_all_cookies()` and `session.erase_session_cookies()`: work with `Cookie` objects. These are ordinary methods, not coroutines.
|
|
140
171
|
- `session.set_response_callback(callback)`: called with each `Response`, including each hop of a redirect.
|
|
172
|
+
- `session.stats()`: the session's request and connection counts. See [Statistics](#statistics).
|
|
173
|
+
- `session.engine`: the session's `Engine`.
|
|
141
174
|
|
|
142
175
|
### Request arguments
|
|
143
176
|
|
|
@@ -149,7 +182,7 @@ An engine binds to the asyncio loop running when it is first used, and so does a
|
|
|
149
182
|
| `cookies` | Dict of cookies to send with this request (added to the session jar). |
|
|
150
183
|
| `cookie_list` | List of `Cookie` objects. |
|
|
151
184
|
| `auth` | `(username, password)` for basic auth. |
|
|
152
|
-
| `data` | Request body: `str` or `bytes` as is, with no `Content-Type` unless you set one, or a dict or list of pairs, form-encoded with `Content-Type: application/x-www-form-urlencoded` unless you set it. |
|
|
185
|
+
| `data` | Request body: `str` or `bytes` as is, with no `Content-Type` unless you set one, or a dict or list of pairs, form-encoded with `Content-Type: application/x-www-form-urlencoded` unless you set it. A file, or another iterable of `bytes` (sync or async), is streamed: see [Large bodies](#large-bodies). |
|
|
153
186
|
| `json` | Object to send as JSON, encoded with the session's `json_dumps`. Sets `Content-Type: application/json` unless you set it. |
|
|
154
187
|
| `files` | Dict (or list of pairs) of files to send as `multipart/form-data`, with any `data` dict as extra fields. A value is an open file, or `(filename, content)` or `(filename, content, content_type)`, where content is `bytes`, `str` or a file. Files are read into memory. |
|
|
155
188
|
| `cert` | `(certfile, keyfile)`, overriding the session's certificate. |
|
|
@@ -157,9 +190,72 @@ An engine binds to the asyncio loop running when it is first used, and so does a
|
|
|
157
190
|
| `timeout`, `connect_timeout` | Override the session's. `timeout=0` removes the session's limit for this request. |
|
|
158
191
|
| `allow_redirects` | Follow redirects (default `True`; `False` for `head`). |
|
|
159
192
|
| `max_redirects` | Default 5. Raises `RequestError("Max Redirects")` beyond it. |
|
|
193
|
+
| `discard_body` | `True` drops the response body as it arrives, for when only the status, headers or timings matter. See [Large bodies](#large-bodies). |
|
|
194
|
+
| `stream` | `True` returns as soon as the headers arrive, and the body is read with `iter_content()` or `read()`. See [Large bodies](#large-bodies). |
|
|
195
|
+
|
|
196
|
+
Request bodies are sent without `Expect: 100-continue`, as in requests, unless you set that header. curl would otherwise add it to bodies over 1 MB and wait up to a second for the server to reply `100 Continue` before sending the body.
|
|
160
197
|
|
|
161
198
|
Failed transfers raise `httpperf.RequestError` with curl's detailed error message, such as `Failed to connect to 127.0.0.1 port 8080 after 0 ms: Couldn't connect to server`. Its `code` is curl's error code. Timeouts raise `httpperf.Timeout`, a subclass, so they can be counted separately.
|
|
162
199
|
|
|
200
|
+
### Large bodies
|
|
201
|
+
|
|
202
|
+
By default the whole response body is kept in memory. For large downloads there are two other ways.
|
|
203
|
+
|
|
204
|
+
`discard_body=True` drops the body on the curl thread as it arrives. Use it when only the status, headers or timings matter; it is the fastest way:
|
|
205
|
+
|
|
206
|
+
```python
|
|
207
|
+
r = await session.get(url, discard_body=True)
|
|
208
|
+
r.status_code, r.download_size, r.total_time
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
`r.body` is then `None`, and `r.text` and `r.json()` raise `ValueError`. `r.download_size` gives the body's size. When a redirect is followed, every response's body is dropped, including those in `history`.
|
|
212
|
+
|
|
213
|
+
`stream=True` works as in requests, with `async for`. The request returns once the headers arrive, and the body is read as it comes:
|
|
214
|
+
|
|
215
|
+
```python
|
|
216
|
+
with open("big.iso", "wb") as f:
|
|
217
|
+
async with await session.get(url, stream=True) as r:
|
|
218
|
+
async for chunk in r.iter_content(1024 * 1024):
|
|
219
|
+
f.write(chunk)
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
- `iter_content(chunk_size)` gives what has arrived, in chunks of at most `chunk_size` bytes. It doesn't wait to fill a chunk, so chunks are often smaller. Without a size, each chunk is everything that has arrived since the last one. `await r.read()` reads the whole body into `r.body`.
|
|
223
|
+
- `r.body`, `r.text` and `r.json()` raise `RuntimeError` until the body has been read, as do the timings and sizes, since they aren't known until the transfer ends.
|
|
224
|
+
- httpperf buffers up to 1 MB of each response. When Python falls behind, it stops reading that connection until Python catches up, so the server is held up by TCP back-pressure. Other requests carry on, and the curl thread never waits. The time spent stopped is `r.paused_time`.
|
|
225
|
+
- Closing the response, leaving `async with`, breaking out of `iter_content()` or dropping the response stops the transfer and closes its connection. Reading to the end leaves the connection open for reuse.
|
|
226
|
+
- A transfer that fails part way, say because the connection drops, raises `RequestError` from `iter_content()` or `read()`.
|
|
227
|
+
- Redirects are followed as usual, and each redirect's body is read into its `history` entry.
|
|
228
|
+
|
|
229
|
+
Uploads stream the same way. Pass an open file, or any object with `read()`, as `data`, or an iterable or async iterable of `bytes`:
|
|
230
|
+
|
|
231
|
+
```python
|
|
232
|
+
with open("big.iso", "rb") as f:
|
|
233
|
+
r = await session.put(url, data=f)
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
- A regular file is sent with a `Content-Length`. Anything else is sent chunked (on HTTP/1.1).
|
|
237
|
+
- httpperf reads ahead up to 1 MB. When the source falls behind, curl pauses that transfer until there is more to send; that time is in `r.paused_time` too.
|
|
238
|
+
- An exception from the source, such as a failing `read()`, stops the transfer and is raised from the request.
|
|
239
|
+
- A streamed upload is never sent twice. If a reused connection turns out to have been closed by the server, the request raises `RequestError`, and it's up to the caller to retry (see below). Bodies given as `bytes` are resent by curl, which it does for any request whose reused connection had closed.
|
|
240
|
+
- A 307 or 308 redirect sends the body again: a seekable file is rewound to where it started, and anything else raises `RequestError`.
|
|
241
|
+
|
|
242
|
+
To retry a streamed upload, open the source again for each attempt:
|
|
243
|
+
|
|
244
|
+
```python
|
|
245
|
+
async def put_file(session, url, path, attempts=2):
|
|
246
|
+
for attempt in range(attempts):
|
|
247
|
+
with open(path, "rb") as f:
|
|
248
|
+
try:
|
|
249
|
+
return await session.put(url, data=f)
|
|
250
|
+
except httpperf.RequestError:
|
|
251
|
+
if attempt == attempts - 1:
|
|
252
|
+
raise
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
`RequestError` includes `Timeout`; put an `except httpperf.Timeout: raise` first if a slow server shouldn't be retried. In a benchmark, count the failed attempts too: each one is a real error, even if the retry succeeded.
|
|
256
|
+
|
|
257
|
+
Streaming is for benchmarking large transfers, such as how fast a server sends or receives a big file, and for checking what arrives. Each chunk passes through Python, so it can't sustain high request rates. For those, use `discard_body=True` or keep the body in memory.
|
|
258
|
+
|
|
163
259
|
### Response
|
|
164
260
|
|
|
165
261
|
| Attribute | Meaning |
|
|
@@ -174,7 +270,8 @@ Failed transfers raise `httpperf.RequestError` with curl's detailed error messag
|
|
|
174
270
|
| `history` | Earlier responses in a redirect chain. |
|
|
175
271
|
| `http_version` | `"HTTP/1.0"`, `"HTTP/1.1"`, `"HTTP/2"` or `"HTTP/3"`. |
|
|
176
272
|
| `headers` | Case-insensitive mapping of the response headers. Repeated headers are joined with `, `, as in requests. `headers_tuple` keeps repeats and order. Both skip interim responses such as `100 Continue`. `header` is the raw text of every header block, decoded as latin-1. |
|
|
177
|
-
| `body` | Bytes
|
|
273
|
+
| `body` | Bytes, or `None` if the request used `discard_body`. |
|
|
274
|
+
| `iter_content(chunk_size=None)`, `read()`, `close()` | Read a streamed body in chunks or whole, or stop it. See [Large bodies](#large-bodies). |
|
|
178
275
|
| `text` | Body decoded with `encoding`. |
|
|
179
276
|
| `encoding` | From the `Content-Type` charset, else latin-1 (`utf-8` for JSON). Can be set. |
|
|
180
277
|
| `json()` | Body parsed with the session's `json_loads`, once. By default the standard library, always as UTF-8, as RFC 8259 requires. |
|
|
@@ -217,7 +314,7 @@ All in seconds.
|
|
|
217
314
|
|
|
218
315
|
| Attribute | Meaning |
|
|
219
316
|
|---|---|
|
|
220
|
-
| `start_time`, `end_time` | Wall-clock times (`time.time()`) when the request was made and when Python received the response. |
|
|
317
|
+
| `start_time`, `end_time` | Wall-clock times (`time.time()`) when the request was made and when Python received the response. With `stream=True`, `end_time` is when Python received the last of the body. |
|
|
221
318
|
| `namelookup_time` | DNS, measured by curl from the start of the transfer. |
|
|
222
319
|
| `connect_time` | TCP connect complete. 0 when the connection was reused. |
|
|
223
320
|
| `appconnect_time` | TLS handshake complete. 0 for plain http and reused connections. |
|
|
@@ -226,16 +323,46 @@ All in seconds.
|
|
|
226
323
|
| `starttransfer_time` | First byte received. |
|
|
227
324
|
| `total_time` | Transfer complete. |
|
|
228
325
|
| `queue_time` | Time waiting in httpperf's queue for a connection, under the engine or session limits. 0 if the request did not wait. curl's timings start after it. |
|
|
326
|
+
| `paused_time` | Time the transfer was stopped waiting on Python: to read a streamed body, or to supply more of a streamed upload. It is part of `total_time`. 0 otherwise. |
|
|
229
327
|
| `client_lag` | `end_time − start_time − total_time − queue_time`: time the request spent waiting on Python, before it was sent and after it finished. Measured with the monotonic clock, so clock changes don't affect it. |
|
|
230
328
|
|
|
231
329
|
curl measures its timings on the curl thread, so a busy Python process does not inflate them. `client_lag` is how you see that Python is the bottleneck: if it grows, the client cannot keep up, and achieved request rates describe the client rather than the server. Report it alongside response times (p50, p99, max).
|
|
232
330
|
|
|
233
331
|
`queue_time` is real waiting that a user would see (a browser's "stalled" time), but it is client-side, so keep it out of server latency.
|
|
234
332
|
|
|
333
|
+
## Statistics
|
|
334
|
+
|
|
335
|
+
`session.stats()` and `engine.stats()` return a dict of counts since the session or engine was created. The engine's counts cover all its sessions. Reading them is cheap, so a load test can sample them every second and report the differences as rates.
|
|
336
|
+
|
|
337
|
+
| Key | Meaning |
|
|
338
|
+
|---|---|
|
|
339
|
+
| `requests` | Requests that got a response, whatever its status. Each hop of a redirect counts. |
|
|
340
|
+
| `errors` | Requests that failed with `RequestError`, including timeouts. |
|
|
341
|
+
| `cancelled` | Requests cancelled before they finished. |
|
|
342
|
+
| `http_versions` | `requests` by HTTP version: a dict with keys `"HTTP/1.0"`, `"HTTP/1.1"`, `"HTTP/2"` and `"HTTP/3"`. |
|
|
343
|
+
| `connections` | New connections opened. A reused connection doesn't count. |
|
|
344
|
+
| `tls_handshakes` | New connections that did a TLS handshake. |
|
|
345
|
+
| `bytes_sent`, `bytes_received` | Application-layer bytes only: request and response headers and bodies, with bodies as transferred (so compressed if the server compressed them). TLS, HTTP/2 and HTTP/3 framing, and TCP/IP overheads aren't counted. |
|
|
346
|
+
| `running` | Requests with curl now. |
|
|
347
|
+
| `queued` | Requests now waiting for a connection under the engine or session limits. |
|
|
348
|
+
| `connections_open` | Connections open now, idle or in use, including any still connecting. For the engine, every pool. For a session with `connections="session"`, its own pool, and 0 once it is closed. `None` for other sessions, whose connections belong to the engine. |
|
|
349
|
+
|
|
350
|
+
The counts are updated on the curl thread, so they can lag a moment behind: a cancelled request is counted just after the cancel.
|
|
351
|
+
|
|
352
|
+
`engine.host_stats()` gives the same counts for each host the engine has sent requests to, keyed by origin, such as `"https://api.example.com:443"`. Different spellings of one host, such as an upper-case scheme or a default port left out, share an entry. `connections_open` is `None` per host. Host stats are off by default; turn them on with `Engine(host_stats=True)`. They cost nothing measurable, but every host is kept until the engine goes, so a client sending requests to a very large number of hosts grows by a few hundred bytes for each.
|
|
353
|
+
|
|
354
|
+
```python
|
|
355
|
+
engine = httpperf.Engine(host_stats=True)
|
|
356
|
+
...
|
|
357
|
+
for origin, counts in engine.host_stats().items():
|
|
358
|
+
print(origin, counts["requests"], counts["errors"])
|
|
359
|
+
```
|
|
360
|
+
|
|
235
361
|
## Known limitations
|
|
236
362
|
|
|
237
363
|
- curl closes expired idle connections lazily, the next time the pool needs a connection, so the server sees the close late. See [docs/connections.md](docs/connections.md).
|
|
238
364
|
- Requests in flight when `Engine.close()` is called never complete.
|
|
365
|
+
- With a system libcurl older than 8.12, `tls="shared"` doesn't share TLS sessions across the engine: curl keeps them per curl handle, so a new connection only resumes if it happens to reuse the handle of an earlier request to that host. The wheels bundle a newer curl.
|
|
239
366
|
|
|
240
367
|
## Development
|
|
241
368
|
|
|
@@ -24,7 +24,7 @@ Bytes are read off the wire as they arrive, whatever Python is doing. If Python
|
|
|
24
24
|
pip install httpperf
|
|
25
25
|
```
|
|
26
26
|
|
|
27
|
-
Wheels are published for Linux on x86_64 and aarch64, for Python 3.9 to 3.15, with glibc 2.34 or later (Ubuntu 22.04, Debian 12, RHEL 9 and later) or musl (Alpine). They include their own libcurl, with HTTP/2 and HTTP/3.
|
|
27
|
+
Wheels are published for Linux on x86_64 and aarch64, for Python 3.9 to 3.15 and free-threaded 3.14 and 3.15 (`python3.14t`, `python3.15t`), with glibc 2.34 or later (Ubuntu 22.04, Debian 12, RHEL 9 and later) or musl (Alpine). They include their own libcurl, with HTTP/2 and HTTP/3.
|
|
28
28
|
|
|
29
29
|
Anywhere else, including Python 3.8, pip builds from source against the system's libcurl, which needs its development headers and a C compiler. That libcurl decides whether HTTP/3 is available: `curl -V` lists `HTTP3` under Features when it does.
|
|
30
30
|
|
|
@@ -60,6 +60,22 @@ asyncio.run(main())
|
|
|
60
60
|
|
|
61
61
|
Sessions are cheap: create one per virtual user or journey. They share a default `Engine` (the curl thread, the shared connection pool and the TLS cache) unless you pass your own.
|
|
62
62
|
|
|
63
|
+
## Command line
|
|
64
|
+
|
|
65
|
+
`python -m httpperf` is a load generator in the spirit of wrk and wrk2. Find the most a server can handle by trying more requests in flight:
|
|
66
|
+
|
|
67
|
+
```
|
|
68
|
+
python -m httpperf -c 10,50,200,1000 http://localhost:8080/
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Then measure latency at fixed rates below that peak:
|
|
72
|
+
|
|
73
|
+
```
|
|
74
|
+
python -m httpperf --rate 20000,40000,60000 http://localhost:8080/
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Each step reports requests per second, latency percentiles, errors and transfer rates, and warns when the client rather than the server was the limit. A short Python script can define the request to make. See [docs/cli.md](docs/cli.md) for all the options.
|
|
78
|
+
|
|
63
79
|
## API
|
|
64
80
|
|
|
65
81
|
### `Session(*, engine=None, ...)`
|
|
@@ -79,7 +95,7 @@ All arguments are keyword-only. A session can be created anywhere, even outside
|
|
|
79
95
|
| `cert` | `None` | `(certfile, keyfile)` paths for a client certificate, used by every request unless the request overrides it. |
|
|
80
96
|
| `timeout` | `None` | Seconds allowed for each request, from the start of the transfer to the last byte (each redirect hop separately). `None` or 0 means no limit. Cancelling the task (or `asyncio.wait_for`) also stops the transfer and frees its connection slot; on HTTP/1.1 that closes the connection, on HTTP/2 and HTTP/3 it resets only that stream. |
|
|
81
97
|
| `connect_timeout` | `None` | Seconds allowed to connect, TLS included. `None` means curl's default of 300 s. |
|
|
82
|
-
| `verify` | `
|
|
98
|
+
| `verify` | `True` | Server certificate checking. `False`: none, for test servers with self-signed certificates. `True`: check against the system CAs, from the first of: `SSL_CERT_FILE` and `SSL_CERT_DIR`, if set and present; libcurl's own default bundle, if present; Python's default bundle (`ssl.get_default_verify_paths()`); the usual system locations. A path: check against that CA bundle. Requests can override it. |
|
|
83
99
|
| `json_dumps` | `None` | Function that turns a `json=` argument into the request body (`str` or `bytes`). `None` means the engine's. |
|
|
84
100
|
| `json_loads` | `None` | Function that turns the response body (`bytes`) into the result of `json()`. `None` means the engine's. |
|
|
85
101
|
|
|
@@ -87,7 +103,7 @@ Proxies come from the environment, as curl reads them: `http_proxy` (not `HTTP_P
|
|
|
87
103
|
|
|
88
104
|
Cookies always belong to the session, whatever the connection settings.
|
|
89
105
|
|
|
90
|
-
### `Engine(*, max_host_connections=0, max_total_connections=0, max_idle_connections=1000, json_dumps=..., json_loads=...)`
|
|
106
|
+
### `Engine(*, max_host_connections=0, max_total_connections=0, max_idle_connections=1000, buffer_size=16384, host_stats=False, json_dumps=..., json_loads=...)`
|
|
91
107
|
|
|
92
108
|
What sessions share: the curl thread, the shared connection pool, the TLS session cache and the limits. Most programs never create one. Create one to set limits, or to keep groups of sessions in separate pools, and pass it to `Session(engine=...)`.
|
|
93
109
|
|
|
@@ -96,14 +112,29 @@ What sessions share: the curl thread, the shared connection pool, the TLS sessio
|
|
|
96
112
|
| `max_host_connections` | Cap on connections per host for the engine's sessions that use the shared pool or no pool. 0 means no limit. httpperf queues requests above it and reports the wait as `queue_time`. |
|
|
97
113
|
| `max_total_connections` | Cap on all connections for the same sessions. 0 means no limit. |
|
|
98
114
|
| `max_idle_connections` | How many connections each pool keeps (the shared pool, and each session's own pool). Must be at least 1. |
|
|
115
|
+
| `buffer_size` | Bytes curl reads from a connection at a time, from 1024 to 10 MB. The default, curl's own, suits many small responses. For large downloads 256 KB is much faster (here 14 to 23 GB/s with `discard_body`, 10 to 15 GB/s streamed), but each transfer in flight has its own buffer, so it costs memory at high connection counts. |
|
|
116
|
+
| `host_stats` | `True` keeps counts for each host, read with `engine.host_stats()`. See [Statistics](#statistics). |
|
|
99
117
|
| `json_dumps`, `json_loads` | The JSON functions for the engine's sessions, unless a session sets its own. The defaults use the standard library: compact output, and bodies decoded as UTF-8. See [JSON](#json). |
|
|
100
118
|
|
|
101
119
|
Each asyncio loop has its own default engine, so sessions without an engine work in any thread, and in each `asyncio.run()`, without setup. A loop's default engine lasts as long as the loop, and is closed once the loop has been closed and no session uses it.
|
|
102
120
|
|
|
103
121
|
An engine binds to the asyncio loop running when it is first used, and so does a session. An engine serves one loop at a time, and using it from a second live loop raises `RuntimeError`. Once its loop is closed, as at the end of `asyncio.run()`, it binds to the next one and keeps its pool. So a session kept between `asyncio.run()` calls keeps its connections. To run loops in several threads at once with your own engines, give each thread its own `Engine`.
|
|
104
122
|
|
|
123
|
+
On free-threaded Python (3.14t and later), httpperf runs without the GIL, so threads that each run their own loop and engine run their Python code in parallel. That is what helps when Python work on each response is the limit, as it usually is in a real load test. On 4 cores, with 20 µs of Python work per response, 4 threads went from 42k requests per second with the GIL to 111k without it; with 100 µs of work, from 9.6k to 34k. With little Python work per response the GIL costs little, since the curl threads never hold it.
|
|
124
|
+
|
|
105
125
|
`engine.close()` stops the curl thread and closes the shared pool's connections, and later requests raise `RuntimeError`. Requests still in flight fail with `RequestError`, and a `ResourceWarning` says how many there were. `with` and `async with` call `close()` on exit. An engine that is no longer referenced is closed automatically. See [docs/connections.md](docs/connections.md) for how these fit together.
|
|
106
126
|
|
|
127
|
+
`engine.stats()` gives the counts for all the engine's sessions. See [Statistics](#statistics).
|
|
128
|
+
|
|
129
|
+
Each engine has one curl thread. When the Python code does real work on each response, Python is the limit and one engine is plenty. A simple load driver that does little per response can make the curl thread the limit instead. Several engines can share one asyncio loop, so spreading sessions over them drives more throughput:
|
|
130
|
+
|
|
131
|
+
```python
|
|
132
|
+
engines = [httpperf.Engine() for _ in range(4)]
|
|
133
|
+
sessions = [httpperf.Session(engine=engines[i % len(engines)]) for i in range(100)]
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
Each engine has its own connections and limits. On 4 cores against a local server, 4 engines took a closed loop of 100 requests from 104k to 223k requests per second for small responses, and from 60k to 183k for 100 KB ones, at which point the Python thread was the limit.
|
|
137
|
+
|
|
107
138
|
### Session methods
|
|
108
139
|
|
|
109
140
|
- `await session.get(url, **kwargs)`, and likewise `post`, `put`, `patch`, `delete`, `head` and `options`.
|
|
@@ -112,6 +143,8 @@ An engine binds to the asyncio loop running when it is first used, and so does a
|
|
|
112
143
|
- `session.cookies()`: the cookie jar as a dict.
|
|
113
144
|
- `session.get_cookie_list()`, `session.add_cookie_list(cookies)`, `session.erase_all_cookies()` and `session.erase_session_cookies()`: work with `Cookie` objects. These are ordinary methods, not coroutines.
|
|
114
145
|
- `session.set_response_callback(callback)`: called with each `Response`, including each hop of a redirect.
|
|
146
|
+
- `session.stats()`: the session's request and connection counts. See [Statistics](#statistics).
|
|
147
|
+
- `session.engine`: the session's `Engine`.
|
|
115
148
|
|
|
116
149
|
### Request arguments
|
|
117
150
|
|
|
@@ -123,7 +156,7 @@ An engine binds to the asyncio loop running when it is first used, and so does a
|
|
|
123
156
|
| `cookies` | Dict of cookies to send with this request (added to the session jar). |
|
|
124
157
|
| `cookie_list` | List of `Cookie` objects. |
|
|
125
158
|
| `auth` | `(username, password)` for basic auth. |
|
|
126
|
-
| `data` | Request body: `str` or `bytes` as is, with no `Content-Type` unless you set one, or a dict or list of pairs, form-encoded with `Content-Type: application/x-www-form-urlencoded` unless you set it. |
|
|
159
|
+
| `data` | Request body: `str` or `bytes` as is, with no `Content-Type` unless you set one, or a dict or list of pairs, form-encoded with `Content-Type: application/x-www-form-urlencoded` unless you set it. A file, or another iterable of `bytes` (sync or async), is streamed: see [Large bodies](#large-bodies). |
|
|
127
160
|
| `json` | Object to send as JSON, encoded with the session's `json_dumps`. Sets `Content-Type: application/json` unless you set it. |
|
|
128
161
|
| `files` | Dict (or list of pairs) of files to send as `multipart/form-data`, with any `data` dict as extra fields. A value is an open file, or `(filename, content)` or `(filename, content, content_type)`, where content is `bytes`, `str` or a file. Files are read into memory. |
|
|
129
162
|
| `cert` | `(certfile, keyfile)`, overriding the session's certificate. |
|
|
@@ -131,9 +164,72 @@ An engine binds to the asyncio loop running when it is first used, and so does a
|
|
|
131
164
|
| `timeout`, `connect_timeout` | Override the session's. `timeout=0` removes the session's limit for this request. |
|
|
132
165
|
| `allow_redirects` | Follow redirects (default `True`; `False` for `head`). |
|
|
133
166
|
| `max_redirects` | Default 5. Raises `RequestError("Max Redirects")` beyond it. |
|
|
167
|
+
| `discard_body` | `True` drops the response body as it arrives, for when only the status, headers or timings matter. See [Large bodies](#large-bodies). |
|
|
168
|
+
| `stream` | `True` returns as soon as the headers arrive, and the body is read with `iter_content()` or `read()`. See [Large bodies](#large-bodies). |
|
|
169
|
+
|
|
170
|
+
Request bodies are sent without `Expect: 100-continue`, as in requests, unless you set that header. curl would otherwise add it to bodies over 1 MB and wait up to a second for the server to reply `100 Continue` before sending the body.
|
|
134
171
|
|
|
135
172
|
Failed transfers raise `httpperf.RequestError` with curl's detailed error message, such as `Failed to connect to 127.0.0.1 port 8080 after 0 ms: Couldn't connect to server`. Its `code` is curl's error code. Timeouts raise `httpperf.Timeout`, a subclass, so they can be counted separately.
|
|
136
173
|
|
|
174
|
+
### Large bodies
|
|
175
|
+
|
|
176
|
+
By default the whole response body is kept in memory. For large downloads there are two other ways.
|
|
177
|
+
|
|
178
|
+
`discard_body=True` drops the body on the curl thread as it arrives. Use it when only the status, headers or timings matter; it is the fastest way:
|
|
179
|
+
|
|
180
|
+
```python
|
|
181
|
+
r = await session.get(url, discard_body=True)
|
|
182
|
+
r.status_code, r.download_size, r.total_time
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
`r.body` is then `None`, and `r.text` and `r.json()` raise `ValueError`. `r.download_size` gives the body's size. When a redirect is followed, every response's body is dropped, including those in `history`.
|
|
186
|
+
|
|
187
|
+
`stream=True` works as in requests, with `async for`. The request returns once the headers arrive, and the body is read as it comes:
|
|
188
|
+
|
|
189
|
+
```python
|
|
190
|
+
with open("big.iso", "wb") as f:
|
|
191
|
+
async with await session.get(url, stream=True) as r:
|
|
192
|
+
async for chunk in r.iter_content(1024 * 1024):
|
|
193
|
+
f.write(chunk)
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
- `iter_content(chunk_size)` gives what has arrived, in chunks of at most `chunk_size` bytes. It doesn't wait to fill a chunk, so chunks are often smaller. Without a size, each chunk is everything that has arrived since the last one. `await r.read()` reads the whole body into `r.body`.
|
|
197
|
+
- `r.body`, `r.text` and `r.json()` raise `RuntimeError` until the body has been read, as do the timings and sizes, since they aren't known until the transfer ends.
|
|
198
|
+
- httpperf buffers up to 1 MB of each response. When Python falls behind, it stops reading that connection until Python catches up, so the server is held up by TCP back-pressure. Other requests carry on, and the curl thread never waits. The time spent stopped is `r.paused_time`.
|
|
199
|
+
- Closing the response, leaving `async with`, breaking out of `iter_content()` or dropping the response stops the transfer and closes its connection. Reading to the end leaves the connection open for reuse.
|
|
200
|
+
- A transfer that fails part way, say because the connection drops, raises `RequestError` from `iter_content()` or `read()`.
|
|
201
|
+
- Redirects are followed as usual, and each redirect's body is read into its `history` entry.
|
|
202
|
+
|
|
203
|
+
Uploads stream the same way. Pass an open file, or any object with `read()`, as `data`, or an iterable or async iterable of `bytes`:
|
|
204
|
+
|
|
205
|
+
```python
|
|
206
|
+
with open("big.iso", "rb") as f:
|
|
207
|
+
r = await session.put(url, data=f)
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
- A regular file is sent with a `Content-Length`. Anything else is sent chunked (on HTTP/1.1).
|
|
211
|
+
- httpperf reads ahead up to 1 MB. When the source falls behind, curl pauses that transfer until there is more to send; that time is in `r.paused_time` too.
|
|
212
|
+
- An exception from the source, such as a failing `read()`, stops the transfer and is raised from the request.
|
|
213
|
+
- A streamed upload is never sent twice. If a reused connection turns out to have been closed by the server, the request raises `RequestError`, and it's up to the caller to retry (see below). Bodies given as `bytes` are resent by curl, which it does for any request whose reused connection had closed.
|
|
214
|
+
- A 307 or 308 redirect sends the body again: a seekable file is rewound to where it started, and anything else raises `RequestError`.
|
|
215
|
+
|
|
216
|
+
To retry a streamed upload, open the source again for each attempt:
|
|
217
|
+
|
|
218
|
+
```python
|
|
219
|
+
async def put_file(session, url, path, attempts=2):
|
|
220
|
+
for attempt in range(attempts):
|
|
221
|
+
with open(path, "rb") as f:
|
|
222
|
+
try:
|
|
223
|
+
return await session.put(url, data=f)
|
|
224
|
+
except httpperf.RequestError:
|
|
225
|
+
if attempt == attempts - 1:
|
|
226
|
+
raise
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
`RequestError` includes `Timeout`; put an `except httpperf.Timeout: raise` first if a slow server shouldn't be retried. In a benchmark, count the failed attempts too: each one is a real error, even if the retry succeeded.
|
|
230
|
+
|
|
231
|
+
Streaming is for benchmarking large transfers, such as how fast a server sends or receives a big file, and for checking what arrives. Each chunk passes through Python, so it can't sustain high request rates. For those, use `discard_body=True` or keep the body in memory.
|
|
232
|
+
|
|
137
233
|
### Response
|
|
138
234
|
|
|
139
235
|
| Attribute | Meaning |
|
|
@@ -148,7 +244,8 @@ Failed transfers raise `httpperf.RequestError` with curl's detailed error messag
|
|
|
148
244
|
| `history` | Earlier responses in a redirect chain. |
|
|
149
245
|
| `http_version` | `"HTTP/1.0"`, `"HTTP/1.1"`, `"HTTP/2"` or `"HTTP/3"`. |
|
|
150
246
|
| `headers` | Case-insensitive mapping of the response headers. Repeated headers are joined with `, `, as in requests. `headers_tuple` keeps repeats and order. Both skip interim responses such as `100 Continue`. `header` is the raw text of every header block, decoded as latin-1. |
|
|
151
|
-
| `body` | Bytes
|
|
247
|
+
| `body` | Bytes, or `None` if the request used `discard_body`. |
|
|
248
|
+
| `iter_content(chunk_size=None)`, `read()`, `close()` | Read a streamed body in chunks or whole, or stop it. See [Large bodies](#large-bodies). |
|
|
152
249
|
| `text` | Body decoded with `encoding`. |
|
|
153
250
|
| `encoding` | From the `Content-Type` charset, else latin-1 (`utf-8` for JSON). Can be set. |
|
|
154
251
|
| `json()` | Body parsed with the session's `json_loads`, once. By default the standard library, always as UTF-8, as RFC 8259 requires. |
|
|
@@ -191,7 +288,7 @@ All in seconds.
|
|
|
191
288
|
|
|
192
289
|
| Attribute | Meaning |
|
|
193
290
|
|---|---|
|
|
194
|
-
| `start_time`, `end_time` | Wall-clock times (`time.time()`) when the request was made and when Python received the response. |
|
|
291
|
+
| `start_time`, `end_time` | Wall-clock times (`time.time()`) when the request was made and when Python received the response. With `stream=True`, `end_time` is when Python received the last of the body. |
|
|
195
292
|
| `namelookup_time` | DNS, measured by curl from the start of the transfer. |
|
|
196
293
|
| `connect_time` | TCP connect complete. 0 when the connection was reused. |
|
|
197
294
|
| `appconnect_time` | TLS handshake complete. 0 for plain http and reused connections. |
|
|
@@ -200,16 +297,46 @@ All in seconds.
|
|
|
200
297
|
| `starttransfer_time` | First byte received. |
|
|
201
298
|
| `total_time` | Transfer complete. |
|
|
202
299
|
| `queue_time` | Time waiting in httpperf's queue for a connection, under the engine or session limits. 0 if the request did not wait. curl's timings start after it. |
|
|
300
|
+
| `paused_time` | Time the transfer was stopped waiting on Python: to read a streamed body, or to supply more of a streamed upload. It is part of `total_time`. 0 otherwise. |
|
|
203
301
|
| `client_lag` | `end_time − start_time − total_time − queue_time`: time the request spent waiting on Python, before it was sent and after it finished. Measured with the monotonic clock, so clock changes don't affect it. |
|
|
204
302
|
|
|
205
303
|
curl measures its timings on the curl thread, so a busy Python process does not inflate them. `client_lag` is how you see that Python is the bottleneck: if it grows, the client cannot keep up, and achieved request rates describe the client rather than the server. Report it alongside response times (p50, p99, max).
|
|
206
304
|
|
|
207
305
|
`queue_time` is real waiting that a user would see (a browser's "stalled" time), but it is client-side, so keep it out of server latency.
|
|
208
306
|
|
|
307
|
+
## Statistics
|
|
308
|
+
|
|
309
|
+
`session.stats()` and `engine.stats()` return a dict of counts since the session or engine was created. The engine's counts cover all its sessions. Reading them is cheap, so a load test can sample them every second and report the differences as rates.
|
|
310
|
+
|
|
311
|
+
| Key | Meaning |
|
|
312
|
+
|---|---|
|
|
313
|
+
| `requests` | Requests that got a response, whatever its status. Each hop of a redirect counts. |
|
|
314
|
+
| `errors` | Requests that failed with `RequestError`, including timeouts. |
|
|
315
|
+
| `cancelled` | Requests cancelled before they finished. |
|
|
316
|
+
| `http_versions` | `requests` by HTTP version: a dict with keys `"HTTP/1.0"`, `"HTTP/1.1"`, `"HTTP/2"` and `"HTTP/3"`. |
|
|
317
|
+
| `connections` | New connections opened. A reused connection doesn't count. |
|
|
318
|
+
| `tls_handshakes` | New connections that did a TLS handshake. |
|
|
319
|
+
| `bytes_sent`, `bytes_received` | Application-layer bytes only: request and response headers and bodies, with bodies as transferred (so compressed if the server compressed them). TLS, HTTP/2 and HTTP/3 framing, and TCP/IP overheads aren't counted. |
|
|
320
|
+
| `running` | Requests with curl now. |
|
|
321
|
+
| `queued` | Requests now waiting for a connection under the engine or session limits. |
|
|
322
|
+
| `connections_open` | Connections open now, idle or in use, including any still connecting. For the engine, every pool. For a session with `connections="session"`, its own pool, and 0 once it is closed. `None` for other sessions, whose connections belong to the engine. |
|
|
323
|
+
|
|
324
|
+
The counts are updated on the curl thread, so they can lag a moment behind: a cancelled request is counted just after the cancel.
|
|
325
|
+
|
|
326
|
+
`engine.host_stats()` gives the same counts for each host the engine has sent requests to, keyed by origin, such as `"https://api.example.com:443"`. Different spellings of one host, such as an upper-case scheme or a default port left out, share an entry. `connections_open` is `None` per host. Host stats are off by default; turn them on with `Engine(host_stats=True)`. They cost nothing measurable, but every host is kept until the engine goes, so a client sending requests to a very large number of hosts grows by a few hundred bytes for each.
|
|
327
|
+
|
|
328
|
+
```python
|
|
329
|
+
engine = httpperf.Engine(host_stats=True)
|
|
330
|
+
...
|
|
331
|
+
for origin, counts in engine.host_stats().items():
|
|
332
|
+
print(origin, counts["requests"], counts["errors"])
|
|
333
|
+
```
|
|
334
|
+
|
|
209
335
|
## Known limitations
|
|
210
336
|
|
|
211
337
|
- curl closes expired idle connections lazily, the next time the pool needs a connection, so the server sees the close late. See [docs/connections.md](docs/connections.md).
|
|
212
338
|
- Requests in flight when `Engine.close()` is called never complete.
|
|
339
|
+
- With a system libcurl older than 8.12, `tls="shared"` doesn't share TLS sessions across the engine: curl keeps them per curl handle, so a new connection only resumes if it happens to reuse the handle of an earlier request to that host. The wheels bundle a newer curl.
|
|
213
340
|
|
|
214
341
|
## Development
|
|
215
342
|
|