lkrequest 0.1.1__tar.gz → 0.2.1__tar.gz

{lkrequest-0.1.1 → lkrequest-0.2.1}/.gitignore

@@ -18,3 +18,4 @@ EVOLUTION_PLAN.md
 CLAUDE.md
 Makefile
 AGENTS.md
+uv.lock

lkrequest-0.2.1/CHANGELOG.md

@@ -0,0 +1,95 @@
+# Changelog
+
+All important changes to this project will be documented in this file.
+
+## lkrequest-py 0.2.1
+
+Adds the Chrome 152 preset and browser network session policies, and picks up a
+large upstream RFC-compliance pass. Several fixes below change observable
+behaviour — the ECH, retry, redirect, cookie and connection-limit entries are
+the ones most likely to affect existing code.
+
+### 🚀 Features
+- **Chrome 152 Preset** — Added `Client.chrome_152()`, `BlockingClient.chrome_152()`, `TlsProfile.chrome_152()`, `TlsProfile.chrome_152_quic()`, `H2Profile.chrome_152()`, `QuicProfile.chrome_152()`, and the `"chrome_152"` TLS/H2/QUIC profile names. Chrome 152 is currently the only preset that applies GREASE to `signature_algorithms`, and it carries the new trust anchor IDs extension. HTTP/2 is unchanged from Chrome 151; the QUIC capture drops the `google_initial_rtt` transport parameter (`0x3127`).
+- **Browser Network Session Policies** — Added `TlsSessionResumptionPolicy` (`BROWSER_DEFAULT` / `DISABLED` / `enabled(max_tickets_per_server)`), `TlsSessionCachePartitionPolicy` (`UNPARTITIONED` / `TOP_LEVEL_SITE` / `TOP_LEVEL_AND_FRAME_SITE` / `CHROMIUM` / `FIREFOX`), and `NetworkPartitionContext(top_level_site, frame_site)`. Set the first two via `Client(tls_session_resumption_policy=..., tls_session_cache_partition_policy=...)` and pass the third to `session(network_partition_context=...)` to reproduce a browser's per-site ticket isolation. These control whether tickets are *stored* and how the cache is keyed; `SessionResumptionConfig` continues to control which resumption modes the ClientHello *advertises*.
+- **HTTP/2 DATA Frame Policy** — Added `Client(h2_data_frame_policy=...)` with `H2DataFramePolicy.BROWSER_DEFAULT`, `.PEER_MAX_FRAME_SIZE`, `.fixed_payload(n)`, and `.socket_write_aligned(n)`, choosing how request bodies are split into DATA frames. The default keeps the browser preset's native framing, since frame sizes are themselves an observable fingerprint trait.
+- **Strict TLS close_notify** — Added `Client(require_close_notify=True)` and a matching read-only property. When enabled, a response body truncated by a bare socket EOF — with no TLS `close_notify` alert — is reported as an error instead of being accepted as a complete body. One part applies regardless of the flag: the read loop now stops at `close_notify` instead of reading on, because anything arriving after it did not come from the peer — those bytes were previously appended to the response body.
+- **Trust Anchor IDs Extension** — Added `ExtType.TRUST_ANCHOR_IDS` and a new `ExtensionSpec` source: `ExtensionSpec(ExtType.TRUST_ANCHOR_IDS, source="trust_anchor_ids", trust_anchor_ids=[...], shuffle=False)`, with `trust_anchor_ids` and `shuffle` readable back off the spec.
+- **DNS-over-HTTPS Feature** — Upstream moved the DoH resolvers behind a `doh` Cargo feature. The published wheels enable it, so `dns="google_https"`, `"cloudflare_https"` and `"quad9_https"` keep working exactly as before; a reduced-feature source build now rejects those three names with an actionable error instead of a generic "unknown DNS config".
+
+### 🐞 Fixes
+- **A Rejected ECH No Longer Falls Through In The Clear** — when a server did not accept Encrypted ClientHello, the handshake previously carried on and handed back a connection whose real server name had never been protected, so the request ECH existed to hide went out in the open. The handshake now aborts before any traffic keys exist, and the session redials once with the `retry_configs` the server supplied — the same thing a browser does, so routine ECH key rotation stays transparent. Retrying *without* ECH is deliberately not a fallback. This affects clients that never set `ech_config=`, because ECH is enabled automatically from DNS HTTPS records; a rejection that cannot be retried now surfaces as `TlsError`. GREASE ECH is unaffected.
+- **Status-Code Retries Are Gated On Replay Safety** — a non-idempotent request answered with a status in the retry set was replayed even though the answer proves the server received it, so a `POST` answered `502` was sent twice. Such a method is now retried only on a status that says it was *not* applied (`429`, `503`); `502`, `504` and the Cloudflare `520`–`530` range are returned to the caller instead. Safe methods are unaffected, and passing `idempotency=Idempotency.Idempotent` on the request restores the old behaviour per call.
+- **Only Real Redirect Statuses Are Followed** — `300`, `304`, `305` and `306` are no longer treated as redirects. Previously a `304 Not Modified` or a `305 Use Proxy` could send the client chasing a `Location` header it should have ignored. `301`, `302`, `303`, `307` and `308` are unaffected.
+- **Cookie Name Prefixes Enforced** — Cookies named with the `__Secure-` or `__Host-` prefix are now rejected unless they satisfy the prefix rules (`Secure`, and for `__Host-` also a `/` path with no `Domain`). A server can no longer set a lookalike cookie that client code would reasonably trust.
+- **Secure Cookies Ignored Over Plain HTTP** — A `Set-Cookie` carrying `Secure` received over a plain-HTTP channel is now discarded rather than stored.
+- **Sessions Are No Longer Capped At 64 Connections** — The implicit per-session ceiling made unrelated high-concurrency workloads fail with `session connection limit reached`. The default is now unbounded; set `max_connections_per_session` explicitly to impose a budget.
+- **Multi-Layer Content-Encoding Is Decoded** — A response carrying stacked encodings (for example `Content-Encoding: gzip, br`) is now fully decoded instead of being returned partly compressed. The zlib-wrapped form of `deflate` is also accepted, matching what browsers tolerate.
+- **WebSocket Handshakes And Close Codes Are Validated** — the upgrade response was checked only for a `101`, which on its own proves nothing; a wrong or missing `Sec-WebSocket-Accept` merely logged a warning. RFC 6455 §4.1 makes that, a missing or wrong `Upgrade`/`Connection`, and any unoffered extension or subprotocol grounds for failing the connection, and all are enforced now. Enforcing it also uncovered that `Sec-WebSocket-Accept` was being computed with a GUID that is not the one §1.3 specifies, so the token never matched a conforming server — harmless while the check only warned. On the sending side, `1005`, `1006` and `1015` may never appear on the wire (§7.4.1) and are now rejected locally, along with over-long close reasons, instead of producing an invalid frame.
+- **HSTS Is Learned Only From Secure Transport** — `Strict-Transport-Security` is no longer recorded from a plain-HTTP response, and never for IP-literal hosts.
+- **Multipart Bodies Match Chrome** — Boundary generation and parameter escaping now follow Chrome's format. A `Content-Type` containing CR or LF is rejected outright, since — unlike the quoted `Content-Disposition` parameters, which are escaped — it cannot be encoded safely and would be a live header-injection vector.
+- **Redirect Method Rewriting Follows The Spec** — The request method is now rewritten only in the cases the specification calls for, so `307` and `308` correctly preserve the original method and body.
+- **Graceful HTTP/2 GOAWAY Is Honoured** — a graceful shutdown carries `NO_ERROR` and therefore set no connection error, while the draining check keyed off "has an error". The driver kept opening streams on a connection the peer had already promised to refuse. Google and Cloudflare send these routinely for connection turnover, so this showed up as sporadic failures under sustained load; queued requests now fail over to a fresh connection.
+- **Alternative Services No Longer Supply The TLS Identity** — an Alt-Svc entry or DNS HTTPS record says *where* to send packets, but the endpoint must still prove it may serve the origin. The advertised host was used as the SNI and certificate name, so the alternative authenticated as itself. Besides the trust problem this broke conforming deployments outright, since an HTTPS RR `TargetName` is typically a CDN-internal name serving the origin's certificate.
+- **Certificates Compressed With zstd Decompress** — `compress_certificate` advertises zlib, brotli and zstd (the Firefox 147 profile offers all three), but only zlib and brotli could be decoded, so a server that picked zstd failed the handshake. All three are handled now (RFC 8879).
+- **Coalesced TLS Post-Handshake Messages Are Reassembled** — post-handshake messages follow the same framing as the handshake: several may share one record, and one may span several. Each record was treated as exactly one message, so a `KeyUpdate` sharing a record with a `NewSessionTicket` was silently dropped — read keys were never rotated and every record after it failed to decrypt — and coalesced tickets past the first were lost.
+- **Protocol Hardening Across TLS, HTTP/2, QUIC And Alt-Svc** — Picks up a broad upstream compliance pass: TLS 1.2 renegotiation is refused with a fatal alert, the record layer and session-ticket trust are bounded, servers can no longer select parameters the client never offered, HTTP/2 enforces response pseudo-header rules and closes connections on peer protocol errors, `record_size_limit` is honoured when sending (RFC 8449), draft HTTP/3 Alt-Svc advertisements are ignored, and SOCKS5 authentication replies are version-checked.
+
+## lkrequest-py 0.2.0
+
+### 🚀 Features
+- **System DNS Positive Cache** — Added `system_dns_cache_ttl` and `system_dns_cache_max_entries` to `Client` and `BlockingClient`. The OS resolver already coalesces concurrent identical lookups process-wide; these cache successful, non-empty results for `ttl` seconds (default capacity 1024 entries). A `0` TTL keeps the coalescing while disabling the cache. It selects the system resolver, so it cannot be combined with `dns`; errors and empty results are never cached.
+- **Shutdown Draining** — Added `drain_pending(timeout=None)`, its blocking counterpart `blocking_drain_pending(...)`, and `pending_requests()`. Waiting for in-flight work before tearing down an event loop means results are still delivered rather than dropped on arrival. Counts are process-wide and cover every async call, not only requests.
+- **HTTP/2 Pending-Request Bound** — Added `max_pending_h2_requests` to `Client` and `BlockingClient`. HTTP/2 requests that exceed the peer's concurrent-stream limit now wait for a stream slot instead of failing, and this option caps how many may queue. Omit it (the default) for an unbounded queue; `0` is rejected with `ValueError`.
+- **Synthetic Fingerprint Negotiability (`synthetic-fp`)** — Added `NegotiabilityFloor.UNIVERSAL`, `NegotiabilityFloor.PRESET_FAMILY`, `NegotiabilityFloor.custom(...)`, and `Randomize.negotiability(...)` so synthesized ClientHello profiles retain a configurable signature-algorithm compatibility floor.
+- **Multi-hop Proxy Chains** — Added `ProxyConfig.parse_chain(...)` and `.through(...)`, with chain metadata via `hop_count()` and `identity()`. Client sessions, proxy pools, and session pools now accept either proxy URLs or `ProxyConfig` objects.
+- **Mixed Proxy Transports** — HTTP/1.1 and HTTP/2 can traverse ordered combinations of HTTP CONNECT, SOCKS5, and SOCKS5H hops. QUIC/HTTP3 supports all-SOCKS5 chains when the optional `quic-h3` feature is enabled.
+- **QUIC-TLS Profile Presets** — Added `TlsProfile.chrome_146_quic()`, `TlsProfile.chrome_150_quic()`, and `TlsProfile.chrome_151_quic()`, exposing the ClientHello Chrome sends inside QUIC (previously reachable only through the bundled `Client.chrome_1xx()` presets). Pass one to `Client(quic_fingerprint=...)` when building a client manually; HTTP/3 otherwise reuses the main TLS profile.
+- **Chrome 151 Preset** — Added `Client.chrome_151()`, `BlockingClient.chrome_151()`, `TlsProfile.chrome_151()`, `H2Profile.chrome_151()`, `QuicProfile.chrome_151()`, and the `"chrome_151"` TLS/H2/QUIC profile names. Captured from Chrome 151.0.7922.72: TCP TLS and HTTP/2 keep Chrome 150's stable fields, while the QUIC-TLS capture omits the three ML-DSA signature algorithms; QUIC transport parameters and HTTP/3 SETTINGS stay aligned with Chrome 150.
+- **Chrome 150 QUIC/H3 (`quic-h3`)** — `Client.chrome_150()` now uses dedicated Chrome 150 QUIC TLS and HTTP/3 profiles instead of reusing Chrome 146. The preset covers captured signature algorithms, QUIC transport parameters, QPACK limits, navigation `PRIORITY_UPDATE`, connection-ID lengths, and Initial datagram sizing. Added `QuicProfile.chrome_150()` and the `"chrome_150"` QUIC profile name.
+
+### 🐞 Fixes
+- **No More Crashes From Late-Completing Requests** — A request that finished after its event loop had closed used to print a `RuntimeError: Event loop is closed` traceback from whichever worker thread it landed on. Enough of them at once turned into `Fatal Python error: _enter_buffered_busy` and killed the process with SIGABRT — uncatchable, and it took the whole batch down. Results arriving after the loop closes are now dropped silently, since nothing is left to receive them.
+- **Multi-Address Connect Fallback** — Connecting now tries every address a hostname resolves to, racing them Happy Eyeballs style (RFC 8305) with interleaved address families, rather than giving up when the first candidate is unreachable. Applies to direct TCP and to SOCKS5 proxies.
+- **HTTP/2 Stream-Limit Backpressure** — Requests beyond the peer's advertised `MAX_CONCURRENT_STREAMS` now wait asynchronously for a stream slot instead of failing with a local capacity error.
+- **Redirects Over Reused HTTP/1.1 Connections** — Following a redirect no longer fails when the pooled HTTP/1.1 connection was torn down in the meantime. The reuse attempt surfaces as hyper's `operation was canceled`, which is now recognised as a closed connection, so the request is retried on a fresh connection instead of erroring out. Affects any server that hangs up after answering a `3xx` — a common pattern after `POST`.
+- **HTTP/2 `1xx` Interim Responses** — The HTTP/2 driver now consumes `1xx` informational responses — such as the `103 Early Hints` Cloudflare sends ahead of its `200` — and returns the final response that follows, instead of reporting the interim status as the final one. A plain `GET https://www.cloudflare.com/` previously came back as `103`.
+- **Connection Limit Enforcement** — `max_connections` now bounds *physical* connections, counting those still being dialled and those checked out by an in-flight request rather than only what sits idle in the pool. Exceeding the limit raises instead of quietly opening another connection, and an idle HTTP/1.1 connection is evicted first so one origin's idle connections cannot starve another. `PoolStats.total` correspondingly reports physical connections in use rather than pooled entries.
+- **Optional Client Certificate Requests** — A server that asks for a client certificate no longer breaks the handshake. TLS 1.2 and TLS 1.3 now answer a `CertificateRequest` with an empty client `Certificate` message — echoing the request context on TLS 1.3, as RFC 8446 §4.4.2 requires — so endpoints with optional mutual TLS complete the handshake instead of failing.
+- **TLS Handshake Reassembly** — TLS 1.2 and TLS 1.3 handshake messages fragmented across multiple records are now buffered and reassembled correctly, preventing failures such as `truncated handshake message in record` without changing ClientHello or fingerprint output.
+- **SOCKS5 UDP Relay Compatibility** — QUIC-over-SOCKS5 accepts relay replies from the advertised source IP even when the relay uses a different egress port, preventing affected HTTP/3 requests from hanging.
+- **Package Version Reporting** — `lkrequest.__version__` is now read from installed distribution metadata instead of being maintained as a separate hard-coded value.
+- **TLS 1.3 ECH + HelloRetryRequest** — Real ECH CH2 now reuses the ECH configuration, HPKE sender context, Inner random, GREASE values, extension permutation, and fake ECH bytes. HRR confirmation, selected groups, cookie-only retries, invalid extensions, dummy CCS, and local fatal alerts now follow the browser/RFC flow.
+- **ECH + PSK + HRR Binders** — `pre_shared_key` remains exclusively in `ClientHelloInner`; CH1 and CH2 binders now use the correct truncated Inner transcripts, including `message_hash(ClientHelloInner1) + HRR` for CH2.
+- **QUIC/H3 HRR State Transitions** — QUIC revokes 0-RTT stream and packet state after HRR and maps local TLS alerts to QUIC `CRYPTO_ERROR` codes.
+
+## lkrequest-py 0.1.1
+
+### 🐞 Fixes
+- **Project URLs** — `Homepage`, `Repository`, and `Issues` in the package metadata pointed at a repository path that does not exist, so every link on the PyPI project page returned 404. They now point at `github.com/EZ-XLabs/lkrequest-py`.
+- **README Links on PyPI** — the language switcher at the top of both READMEs used relative paths (`README.md` / `README-zh.md`), which PyPI cannot resolve when it renders the long description. Both now use absolute URLs, so switching between English and 简体中文 works from the project page.
+
+### 📝 Documentation
+- **Development Setup** — the setup section now walks through the full local workflow on Linux/macOS and Windows: creating and activating a virtualenv, installing maturin, `pip install -e ".[test,lint]"`, and running the suite via `just test`. An equivalent `uv sync --extra test --extra lint` path is documented alongside it.
+
+## lkrequest-py 0.1.0
+
+### 🚀 Features
+- **TLS Fingerprint Control** — Byte-level ClientHello generation that matches real browsers (Chrome, Firefox, Safari)
+- **HTTP/2 Fingerprint Control** — SETTINGS frame order, pseudo-header order, WINDOW_UPDATE, PRIORITY frames, and per-request priority weights
+- **HTTP/3 + QUIC** — Optional `quic-h3` feature for full HTTP/3 over QUIC with fingerprint-aware Transport Parameters, Alt-Svc auto-discovery, and H2→H3 seamless upgrade
+- **Built-in Browser Presets** — Chrome 131/144/145/146/147/148/149/150, Firefox 133/147, Safari 18/26 (TLS + H2, plus QUIC when `quic-h3` is enabled)
+- **Session Management** — Cookie jars, connection pooling, HTTP/2 multiplexing, and optional QUIC 0-RTT session resumption
+- **Connection Prewarming** — `session.preconnect()` pre-establishes DNS + TCP + TLS + H2 (or, with `quic-h3`, QUIC + H3) connections
+- **Custom DNS Resolver** — Pluggable DNS with DoH support, auto ECH config and H3 hints via HTTPS/SVCB records
+- **Alt-Svc Discovery** — Automatic HTTP/2 → HTTP/3 protocol upgrade via Alt-Svc headers with broken-QUIC fallback
+- **SessionPool** — High-concurrency pool with proxy rotation (round-robin / random)
+- **Proxy Support** — SOCKS5 and HTTP CONNECT with authentication
+- **WebSocket** — HTTP/1.1 Upgrade (RFC 6455) and H2 Extended CONNECT (RFC 8441)
+- **Middleware** — Interceptor chain for request/response modification
+- **Retry Policies** — Exponential backoff, fixed interval, custom strategies; idempotency-aware (non-idempotent methods are not silently replayed after a possibly-already-sent failure unless marked `Idempotency::Idempotent`)
+- **Redirect Control** — `RedirectPolicy::Follow(n)` or `RedirectPolicy::None` for manual handling
+- **Auto-Decompression** — Brotli, gzip, deflate, zstd
+- **Blocking API** — Synchronous wrapper for non-async contexts
+- **Multipart/Form-Data** — File uploads with streaming support
+- **TCP Fingerprint** — JA4T-style TCP SYN fingerprinting