tunnelfetch 1.8.3 → 1.8.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +0 -37
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1047,43 +1047,6 @@ subject DN, and only the one anchor a chain lands on is ever decoded, so startup
|
|
|
1047
1047
|
the 380 KB bundle (133 KB gzipped) and a request that imports but does not use the package costs
|
|
1048
1048
|
0 ms.
|
|
1049
1049
|
|
|
1050
|
-
### Streaming APIs: turn HTTP/2 off
|
|
1051
|
-
|
|
1052
|
-
An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
|
|
1053
|
-
arriving as hundreds of tiny events rather than a large one arriving in a few chunks. Over HTTP/2
|
|
1054
|
-
each of those events is a DATA frame with flow control and a `WINDOW_UPDATE` behind it; over
|
|
1055
|
-
HTTP/1.1 it is a chunked-encoding chunk and nothing else. These APIs do not multiplex, so the
|
|
1056
|
-
cheaper framing simply wins:
|
|
1057
|
-
|
|
1058
|
-
```js
|
|
1059
|
-
new Client({ connect, proxy, http2: false });
|
|
1060
|
-
```
|
|
1061
|
-
|
|
1062
|
-
Measured against a real streaming endpoint through a proxy, 20K tokens in and 8K tokens out:
|
|
1063
|
-
|
|
1064
|
-
| | CPU/request | per 1M requests |
|
|
1065
|
-
| --- | --- | --- |
|
|
1066
|
-
| HTTP/2 | 32 ms | $0.94 |
|
|
1067
|
-
| **HTTP/1.1** | **28 ms** | **$0.86** |
|
|
1068
|
-
| platform `fetch` — reference; it cannot use a proxy | 5 ms | $0.40 |
|
|
1069
|
-
|
|
1070
|
-
**13% for one option, and nothing is given up** — a request that never opens a second stream gains
|
|
1071
|
-
nothing from multiplexing.
|
|
1072
|
-
|
|
1073
|
-
Two things about this shape are worth knowing because they are counter-intuitive:
|
|
1074
|
-
|
|
1075
|
-
**The cost is flat in output length.** 512 tokens and 8000 tokens cost within 2 ms of each other.
|
|
1076
|
-
Events are batched by the server and V8 tiers up inside the request, and the two together flatten
|
|
1077
|
-
the curve completely. An earlier version of this section modelled it as linear from two nearby
|
|
1078
|
-
points and predicted a break-even 15x lower than the truth; do not extrapolate this from a slope.
|
|
1079
|
-
|
|
1080
|
-
**The request body is cheap.** A 20K-token prompt is ~108 KB of JSON to serialise, buffer, encrypt
|
|
1081
|
-
and frame, and it costs 4 ms on h1 or 6 ms on h2 — 15–20% of the request, not the dominant term.
|
|
1082
|
-
|
|
1083
|
-
For scale: at these sizes the model's own bill is about **$0.0088 per request**, so this package's
|
|
1084
|
-
CPU is **0.011%** of what you pay. Turn h2 off because it is free, not because it will show up on an
|
|
1085
|
-
invoice.
|
|
1086
|
-
|
|
1087
1050
|
### Streaming APIs, and what they cost
|
|
1088
1051
|
|
|
1089
1052
|
An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tunnelfetch",
|
|
3
|
-
"version": "1.8.
|
|
3
|
+
"version": "1.8.4",
|
|
4
4
|
"description": "A fetch-shaped HTTP client that can route through HTTP CONNECT / HTTPS / SOCKS5 proxies on runtimes with only raw TCP, such as Cloudflare Workers. Implements TLS in userland because the runtime cannot verify a tunnelled peer.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"fetch",
|