tunnelfetch 1.8.2 → 1.8.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -41
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1047,43 +1047,6 @@ subject DN, and only the one anchor a chain lands on is ever decoded, so startup
|
|
|
1047
1047
|
the 380 KB bundle (133 KB gzipped) and a request that imports but does not use the package costs
|
|
1048
1048
|
0 ms.
|
|
1049
1049
|
|
|
1050
|
-
### Streaming APIs: turn HTTP/2 off
|
|
1051
|
-
|
|
1052
|
-
An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
|
|
1053
|
-
arriving as hundreds of tiny events rather than a large one arriving in a few chunks. Over HTTP/2
|
|
1054
|
-
each of those events is a DATA frame with flow control and a `WINDOW_UPDATE` behind it; over
|
|
1055
|
-
HTTP/1.1 it is a chunked-encoding chunk and nothing else. These APIs do not multiplex, so the
|
|
1056
|
-
cheaper framing simply wins:
|
|
1057
|
-
|
|
1058
|
-
```js
|
|
1059
|
-
new Client({ connect, proxy, http2: false });
|
|
1060
|
-
```
|
|
1061
|
-
|
|
1062
|
-
Measured against a real streaming endpoint through a proxy, 20K tokens in and 8K tokens out:
|
|
1063
|
-
|
|
1064
|
-
| | CPU/request | per 1M requests |
|
|
1065
|
-
| --- | --- | --- |
|
|
1066
|
-
| HTTP/2 | 32 ms | $0.94 |
|
|
1067
|
-
| **HTTP/1.1** | **28 ms** | **$0.86** |
|
|
1068
|
-
| platform `fetch` — reference; it cannot use a proxy | 5 ms | $0.40 |
|
|
1069
|
-
|
|
1070
|
-
**13% for one option, and nothing is given up** — a request that never opens a second stream gains
|
|
1071
|
-
nothing from multiplexing.
|
|
1072
|
-
|
|
1073
|
-
Two things about this shape are worth knowing because they are counter-intuitive:
|
|
1074
|
-
|
|
1075
|
-
**The cost is flat in output length.** 512 tokens and 8000 tokens cost within 2 ms of each other.
|
|
1076
|
-
Events are batched by the server and V8 tiers up inside the request, and the two together flatten
|
|
1077
|
-
the curve completely. An earlier version of this section modelled it as linear from two nearby
|
|
1078
|
-
points and predicted a break-even 15x lower than the truth; do not extrapolate this from a slope.
|
|
1079
|
-
|
|
1080
|
-
**The request body is cheap.** A 20K-token prompt is ~108 KB of JSON to serialise, buffer, encrypt
|
|
1081
|
-
and frame, and it costs 4 ms on h1 or 6 ms on h2 — 15–20% of the request, not the dominant term.
|
|
1082
|
-
|
|
1083
|
-
For scale: at these sizes the model's own bill is about **$0.0088 per request**, so this package's
|
|
1084
|
-
CPU is **0.011%** of what you pay. Turn h2 off because it is free, not because it will show up on an
|
|
1085
|
-
invoice.
|
|
1086
|
-
|
|
1087
1050
|
### Streaming APIs, and what they cost
|
|
1088
1051
|
|
|
1089
1052
|
An SSE response from an LLM API is the opposite shape to everything else measured here: a small body
|
|
@@ -1093,10 +1056,15 @@ that are a footnote for a 4 MB page are the whole bill here.
|
|
|
1093
1056
|
Measured against a real streaming endpoint through a proxy, with the API's own `usage` block as the
|
|
1094
1057
|
token count rather than an estimate from response size:
|
|
1095
1058
|
|
|
1096
|
-
| | per event | per 1M output tokens |
|
|
1097
|
-
| --- | --- | --- |
|
|
1098
|
-
| this package | **250–310 µs** |
|
|
1099
|
-
| platform `fetch` — reference; it cannot use a proxy | ~105 µs | $
|
|
1059
|
+
| | per event | CPU per 1M output tokens | cost per 1M output tokens |
|
|
1060
|
+
| --- | --- | --- | --- |
|
|
1061
|
+
| this package | **250–310 µs** | 250–310 s | **$0.0050–0.0062** |
|
|
1062
|
+
| platform `fetch` — reference; it cannot use a proxy | ~105 µs | 105 s | $0.0021 |
|
|
1063
|
+
|
|
1064
|
+
Those are half a cent per million output tokens against **$0.60** of model charge for the same
|
|
1065
|
+
tokens. An earlier version of this table read "$5,000–6,200" — the same digits with the decimal
|
|
1066
|
+
point six places out, from writing a per-million-*requests* figure into a per-million-*tokens*
|
|
1067
|
+
column. The ratio quoted below was computed correctly and is unaffected; the table was not.
|
|
1100
1068
|
|
|
1101
1069
|
Events map to output tokens roughly 1:1, so a 128K-token completion is about **35 s of CPU**. That
|
|
1102
1070
|
is spread across the minutes the model takes to generate it — utilisation is 2–5%, so it is a
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tunnelfetch",
|
|
3
|
-
"version": "1.8.
|
|
3
|
+
"version": "1.8.4",
|
|
4
4
|
"description": "A fetch-shaped HTTP client that can route through HTTP CONNECT / HTTPS / SOCKS5 proxies on runtimes with only raw TCP, such as Cloudflare Workers. Implements TLS in userland because the runtime cannot verify a tunnelled peer.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"fetch",
|