@browsercore/testing 0.1.3 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -0
- package/captures/DIVERGENCE_REPORT.md +325 -0
- package/captures/IMPERSIONATION_GAPS_REPORT.md +367 -0
- package/captures/README.md +56 -0
- package/captures/REVIEW_FINDINGS.md +71 -0
- package/captures/_probe/analyze.py +67 -0
- package/captures/_probe/capture_bytes.py +127 -0
- package/captures/_probe/chrome131_determinism.py +32 -0
- package/captures/_probe/diagnose_capture.py +59 -0
- package/captures/_probe/golden_analysis.py +91 -0
- package/captures/_probe/inspect_golden.py +55 -0
- package/captures/_probe/inspect_sv.py +16 -0
- package/captures/_probe/make_golden.py +170 -0
- package/captures/_probe/output/chrome100.json +219 -0
- package/captures/_probe/output/chrome101.json +219 -0
- package/captures/_probe/output/chrome104.json +219 -0
- package/captures/_probe/output/chrome107.json +220 -0
- package/captures/_probe/output/chrome110.json +220 -0
- package/captures/_probe/output/chrome116.json +220 -0
- package/captures/_probe/output/chrome119.json +219 -0
- package/captures/_probe/output/chrome120.json +219 -0
- package/captures/_probe/output/chrome123.json +223 -0
- package/captures/_probe/output/chrome124.json +224 -0
- package/captures/_probe/output/chrome131.json +224 -0
- package/captures/_probe/output/chrome131_fresh.json +224 -0
- package/captures/_probe/output/chrome133a.json +224 -0
- package/captures/_probe/output/chrome136.json +224 -0
- package/captures/_probe/output/chrome99.json +219 -0
- package/captures/_probe/output/edge101.json +219 -0
- package/captures/_probe/output/edge99.json +219 -0
- package/captures/_probe/output/firefox133.json +227 -0
- package/captures/_probe/output/firefox135.json +230 -0
- package/captures/_probe/output/safari155.json +210 -0
- package/captures/_probe/output/safari170.json +214 -0
- package/captures/_probe/output/safari170_fresh.json +214 -0
- package/captures/_probe/output/safari180.json +216 -0
- package/captures/_probe/output/safari180_ios.json +216 -0
- package/captures/_probe/output/safari184.json +215 -0
- package/captures/_probe/output/safari260.json +211 -0
- package/captures/_probe/output/summary.json +2520 -0
- package/captures/_probe/parse_existing_settings.py +15 -0
- package/captures/_probe/peetprint.py +39 -0
- package/captures/_probe/probe.py +137 -0
- package/captures/_probe/raw_bytes/chrome-131_client_hello.bin +0 -0
- package/captures/_probe/raw_bytes/chrome-131_client_hello.hex +32 -0
- package/captures/_probe/raw_bytes/firefox-133_client_hello.bin +0 -0
- package/captures/_probe/raw_bytes/firefox-133_client_hello.hex +32 -0
- package/captures/_probe/raw_bytes/safari-17_client_hello.bin +0 -0
- package/captures/_probe/raw_bytes/safari-17_client_hello.hex +32 -0
- package/captures/_probe/relay_capture.py +137 -0
- package/captures/_probe/socks_capture.py +138 -0
- package/captures/_probe/validate_fresh.py +66 -0
- package/captures/_probe/validate_golden.py +97 -0
- package/captures/_probe/verify_capture.py +68 -0
- package/captures/chrome-131/http2/settings.bin +0 -0
- package/captures/chrome-131/http2/settings.meta.json +9 -0
- package/captures/chrome-131/oracle_capture.reference.json +224 -0
- package/captures/chrome-131/tls/client_hello.bin +0 -0
- package/captures/chrome-131/tls/client_hello.meta.json +20 -0
- package/captures/chrome-140/http2/settings.bin +0 -0
- package/captures/chrome-140/http2/settings.meta.json +9 -0
- package/captures/chrome-140/tls/client_hello.bin +0 -0
- package/captures/chrome-140/tls/client_hello.meta.json +12 -0
- package/captures/firefox-128/tls/client_hello.bin +0 -0
- package/captures/firefox-128/tls/client_hello.meta.json +12 -0
- package/captures/firefox-133/http2/settings.bin +0 -0
- package/captures/firefox-133/http2/settings.meta.json +9 -0
- package/captures/firefox-133/oracle_capture.reference.json +227 -0
- package/captures/firefox-133/tls/client_hello.bin +0 -0
- package/captures/firefox-133/tls/client_hello.meta.json +20 -0
- package/captures/safari-17/http2/settings.bin +0 -0
- package/captures/safari-17/http2/settings.meta.json +9 -0
- package/captures/safari-17/oracle_capture.reference.json +214 -0
- package/captures/safari-17/tls/client_hello.bin +0 -0
- package/captures/safari-17/tls/client_hello.meta.json +20 -0
- package/dist/bench/bench.d.ts +22 -6
- package/dist/bench/bench.d.ts.map +1 -1
- package/dist/bench/bench.js +105 -10
- package/dist/bench/bench.js.map +1 -1
- package/dist/captures/manifest.d.ts.map +1 -1
- package/dist/captures/manifest.js +2 -3
- package/dist/captures/manifest.js.map +1 -1
- package/dist/categories/cookies.d.ts +3 -2
- package/dist/categories/cookies.d.ts.map +1 -1
- package/dist/categories/cookies.js +228 -11
- package/dist/categories/cookies.js.map +1 -1
- package/dist/categories/redirects.d.ts +8 -2
- package/dist/categories/redirects.d.ts.map +1 -1
- package/dist/categories/redirects.js +97 -12
- package/dist/categories/redirects.js.map +1 -1
- package/dist/e2e/cert-gen.d.ts +30 -0
- package/dist/e2e/cert-gen.d.ts.map +1 -0
- package/dist/e2e/cert-gen.js +158 -0
- package/dist/e2e/cert-gen.js.map +1 -0
- package/dist/e2e/parse-clienthello.d.ts +177 -0
- package/dist/e2e/parse-clienthello.d.ts.map +1 -0
- package/dist/e2e/parse-clienthello.js +343 -0
- package/dist/e2e/parse-clienthello.js.map +1 -0
- package/dist/e2e/sink-server.d.ts +109 -0
- package/dist/e2e/sink-server.d.ts.map +1 -0
- package/dist/e2e/sink-server.js +292 -0
- package/dist/e2e/sink-server.js.map +1 -0
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js.map +1 -1
- package/dist/fingerprint/index.d.ts +5 -2
- package/dist/fingerprint/index.d.ts.map +1 -1
- package/dist/fingerprint/index.js +3 -2
- package/dist/fingerprint/index.js.map +1 -1
- package/dist/fingerprint/ja3.d.ts.map +1 -1
- package/dist/fingerprint/ja3.js +33 -4
- package/dist/fingerprint/ja3.js.map +1 -1
- package/dist/fingerprint/ja4-errors.d.ts +17 -0
- package/dist/fingerprint/ja4-errors.d.ts.map +1 -0
- package/dist/fingerprint/ja4-errors.js +20 -0
- package/dist/fingerprint/ja4-errors.js.map +1 -0
- package/dist/fingerprint/ja4-reader.d.ts +44 -0
- package/dist/fingerprint/ja4-reader.d.ts.map +1 -0
- package/dist/fingerprint/ja4-reader.js +112 -0
- package/dist/fingerprint/ja4-reader.js.map +1 -0
- package/dist/fingerprint/ja4.d.ts +11 -43
- package/dist/fingerprint/ja4.d.ts.map +1 -1
- package/dist/fingerprint/ja4.js +54 -141
- package/dist/fingerprint/ja4.js.map +1 -1
- package/dist/fingerprint/ja4h.d.ts +46 -0
- package/dist/fingerprint/ja4h.d.ts.map +1 -0
- package/dist/fingerprint/ja4h.js +50 -0
- package/dist/fingerprint/ja4h.js.map +1 -0
- package/dist/golden/golden.d.ts +15 -1
- package/dist/golden/golden.d.ts.map +1 -1
- package/dist/golden/golden.js +35 -22
- package/dist/golden/golden.js.map +1 -1
- package/dist/index.d.ts +17 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +26 -1
- package/dist/index.js.map +1 -1
- package/dist/reference/browser-provider.d.ts +33 -0
- package/dist/reference/browser-provider.d.ts.map +1 -0
- package/dist/reference/browser-provider.js +60 -0
- package/dist/reference/browser-provider.js.map +1 -0
- package/dist/reference/curl-provider.d.ts +40 -0
- package/dist/reference/curl-provider.d.ts.map +1 -0
- package/dist/reference/curl-provider.js +96 -0
- package/dist/reference/curl-provider.js.map +1 -0
- package/dist/reference/dump.d.ts +42 -0
- package/dist/reference/dump.d.ts.map +1 -0
- package/dist/reference/dump.js +102 -0
- package/dist/reference/dump.js.map +1 -0
- package/dist/reference/node-reference.d.ts.map +1 -1
- package/dist/reference/node-reference.js +2 -5
- package/dist/reference/node-reference.js.map +1 -1
- package/dist/reference/reference-errors.d.ts +36 -0
- package/dist/reference/reference-errors.d.ts.map +1 -0
- package/dist/reference/reference-errors.js +43 -0
- package/dist/reference/reference-errors.js.map +1 -0
- package/dist/reference/reference-types.d.ts +33 -0
- package/dist/reference/reference-types.d.ts.map +1 -0
- package/dist/reference/reference-types.js +9 -0
- package/dist/reference/reference-types.js.map +1 -0
- package/dist/reference/reference.d.ts +14 -77
- package/dist/reference/reference.d.ts.map +1 -1
- package/dist/reference/reference.js +43 -207
- package/dist/reference/reference.js.map +1 -1
- package/dist/rfc/rfcTests.d.ts +46 -8
- package/dist/rfc/rfcTests.d.ts.map +1 -1
- package/dist/rfc/rfcTests.js +203 -16
- package/dist/rfc/rfcTests.js.map +1 -1
- package/dist/types.d.ts +2 -1
- package/dist/types.d.ts.map +1 -1
- package/dist/types.js.map +1 -1
- package/dist/utils.d.ts +3 -0
- package/dist/utils.d.ts.map +1 -1
- package/dist/utils.js +11 -5
- package/dist/utils.js.map +1 -1
- package/package.json +61 -55
package/README.md
CHANGED
|
@@ -1,5 +1,10 @@
|
|
|
1
1
|
# @browsercore/testing
|
|
2
2
|
|
|
3
|
+
|
|
4
|
+
[](https://www.npmjs.com/package/@browsercore/testing)
|
|
5
|
+
[](https://github.com/jverneuer/browsercore-testing/blob/main/COVERAGE.md)
|
|
6
|
+
[](https://github.com/jverneuer/browsercore-testing/actions/workflows/ci.yml)
|
|
7
|
+
|
|
3
8
|
Protocol verification — RFC compliance tests, browser golden packet captures,
|
|
4
9
|
integration tests, and benchmarking. Compares generated packets against
|
|
5
10
|
captures from real browsers.
|
|
@@ -0,0 +1,325 @@
|
|
|
1
|
+
# Divergence Report: curl-impersonate (curl_cffi) vs `@browsercore`
|
|
2
|
+
|
|
3
|
+
> Generated: 2026-08-04 by automated probe of `https://tls.peet.ws/api/all` (the TLS/HTTP2 fingerprint oracle).
|
|
4
|
+
> curl_cffi version: **0.13.0** (Python 3.9). Every value below is from an actual curl_cffi call unless marked *[inferred]*.
|
|
5
|
+
>
|
|
6
|
+
> **Scope:** This report is analysis-only. No TypeScript source was edited. New golden captures were generated for 3 profiles (see §6).
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## TL;DR
|
|
11
|
+
|
|
12
|
+
- The two existing golden captures (`chrome-140`, `firefox-128`) are **synthetic stubs**, not real wire data — 96-byte ClientHellos with no GREASE, 2 cipher suites, and sequential `client_random` (`00010203…`). Both stubs hash to the **identical** JA3 `853b03398669dbeffb6116ecd6e6beb6`, proving they are placeholders.
|
|
13
|
+
- **24 real browser profiles** were probed successfully. Real ClientHellos are **517–1841 bytes**, carry 16–21 cipher suites, GREASE, and 15–18 extensions.
|
|
14
|
+
- **Top 5 divergence risks** for `@browsercore` relative to curl-impersonate:
|
|
15
|
+
1. **Extension ordering** is the single biggest fingerprint signal — captured by the `peetprint` string; our profiles must reproduce exact per-profile order (chrome-131 is *non-deterministic* across connections).
|
|
16
|
+
2. **GREASE** (RFC 8701) values and *positions* differ per profile; the stubs have none.
|
|
17
|
+
3. **HTTP/2 SETTINGS** contents/order differ by browser (Chrome: `1,2,4,6`; Firefox: `1,2,4,5`; Safari: `2,4,3`).
|
|
18
|
+
4. **Pseudo-header ordering** differs: Chrome `:method,:authority,:scheme,:path`; Firefox `:method,:path,:authority,:scheme`; Safari `:method,:scheme,:path,:authority`.
|
|
19
|
+
5. **Priority weight/depends_on/exclusive** framing on the HEADERS stream (Chrome `exclusive=1`; Firefox/Safari `exclusive=0`; Safari weight `255` not `256`).
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## 1. Profile Matrix
|
|
24
|
+
|
|
25
|
+
All 24 profiles probed returned HTTP 200 from the oracle. Columns: JA3 hash, JA4 tag, `#ext` (TLS extension count), SETTINGS keys (in order), WINDOW_UPDATE increment.
|
|
26
|
+
|
|
27
|
+
| Profile | JA3 hash | JA4 tag | #ext | SETTINGS (order) | WINDOW_UPDATE | Priority (w/dep/excl) |
|
|
28
|
+
|---|---|---|---|---|---|---|
|
|
29
|
+
| chrome99 | cd08e31494f9531f560d64c695473da9 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,3,4,6 | 15663105 | 256/0/1 |
|
|
30
|
+
| chrome100 | cd08e31494f9531f560d64c695473da9 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,3,4,6 | 15663105 | 256/0/1 |
|
|
31
|
+
| chrome101 | cd08e31494f9531f560d64c695473da9 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,3,4,6 | 15663105 | 256/0/1 |
|
|
32
|
+
| chrome104 | cd08e31494f9531f560d64c695473da9 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,3,4,6 | 15663105 | 256/0/1 |
|
|
33
|
+
| chrome107 | cd08e31494f9531f560d64c695473da9 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,2,3,4,6 | 15663105 | 256/0/1 |
|
|
34
|
+
| chrome110 | cd8c6a677122388552c0681187a3fe11 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,2,3,4,6 | 15663105 | 256/0/1 |
|
|
35
|
+
| chrome116 | 26435bc2fd3213d9d637e77b579f3514 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,2,3,4,6 | 15663105 | 256/0/1 |
|
|
36
|
+
| chrome119 | 0f154298f33629789c57012c70acf64a | t13d1516h2_8daaf6152771_02713d6af862 | 17 | 1,2,4,6 | 15663105 | 256/0/1 |
|
|
37
|
+
| chrome120 | 698f6d684588ddc1217dfb4454916129 | t13d1516h2_8daaf6152771_02713d6af862 | 17 | 1,2,4,6 | 15663105 | 256/0/1 |
|
|
38
|
+
| chrome123 | 04bfdd1b851c9909a91b12f551e11055 | t13d1517h2_8daaf6152771_02713d6af862 | 17 | 1,2,4,6 | 15663105 | 256/0/1 |
|
|
39
|
+
| chrome124 | 9bbc2cb427411039ad61822137225012 | t13d1516h2_8daaf6152771_02713d6af862 | 18 | 1,2,4,6 | 15663105 | 256/0/1 |
|
|
40
|
+
| chrome131 | fb519300321e7e157792ac8d3a77e9ee* | t13d1516h2_8daaf6152771_02713d6af862 | 17-18 | 1,2,4,6 | 15663105 | 256/0/1 |
|
|
41
|
+
| chrome133a | 497a76e5b72c28d4236fa42f207c0b38 | t13d1516h2_8daaf6152771_d8a2da3f94cd | 18 | 1,2,4,6 | 15663105 | 256/0/1 |
|
|
42
|
+
| chrome136 | cb3778b7b74a5c21bca0f76860cc668c | t13d1516h2_8daaf6152771_d8a2da3f94cd | 18 | 1,2,4,6 | 15663105 | 256/0/1 |
|
|
43
|
+
| edge99 | cd08e31494f9531f560d64c695473da9 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,3,4,6 | 15663105 | 256/0/1 |
|
|
44
|
+
| edge101 | cd08e31494f9531f560d64c695473da9 | t13d1516h2_8daaf6152771_f37e75b10bcc | 18 | 1,3,4,6 | 15663105 | 256/0/1 |
|
|
45
|
+
| safari155 | 773906b0efdefa24a7f2b8eb6985bf37 | t13d2014h2_a09f3c656075_874d27d7ca63 | 16 | 4,3 | 10485760 | — |
|
|
46
|
+
| safari170 | 773906b0efdefa24a7f2b8eb6985bf37 | t13d2014h2_a09f3c656075_874d27d7ca63 | 16 | 2,4,3 | 10485760 | 255/0/0 |
|
|
47
|
+
| safari180 | 773906b0efdefa24a7f2b8eb6985bf37 | t13d2014h2_a09f3c656075_7f0f34a4126d | 16 | 2,3,4,8,9 | 10420225 | 256/0/0 |
|
|
48
|
+
| safari184 | 773906b0efdefa24a7f2b8eb6985bf37 | t13d2014h2_a09f3c656075_7f0f34a4126d | 16 | 2,3,4,9 | 10420225 | 256/0/0 |
|
|
49
|
+
| safari180_ios | 773906b0efdefa24a7f2b8eb6985bf37 | t13d2014h2_a09f3c656075_7f0f34a4126d | 16 | 2,3,4,8,9 | 10420225 | 256/0/0 |
|
|
50
|
+
| safari260 | 6618231821d2d40f8d1859f5a43a0307 | t13d2014h2_a09f3c656075_d0a99439f9b1 | 16 | 2,3,4,9 | 10420225 | 256/0/0 |
|
|
51
|
+
| firefox133 | 2d692a4485ca2f5f2b10ecb2d2909ad3 | t13d1716h2_5b57614c22b0_eeeea6562960 | 16 | 1,2,4,5 | 12517377 | 256/0/0 |
|
|
52
|
+
| firefox135 | 6f7889b9fb1a62a9577e685c1fcfa919 | t13d1717h2_5b57614c22b0_3cbfd9057e0d | 17 | 1,2,4,5 | 12517377 | 256/0/0 |
|
|
53
|
+
|
|
54
|
+
> **SETTINGS key legend:** `1`=HEADER_TABLE_SIZE, `2`=ENABLE_PUSH, `3`=MAX_CONCURRENT_STREAMS, `4`=INITIAL_WINDOW_SIZE, `5`=MAX_FRAME_SIZE, `6`=MAX_HEADER_LIST_SIZE, `8`=ENABLE_CONNECT_PROTOCOL, `9`=NO_RFC7540_PRIORITIES.
|
|
55
|
+
>
|
|
56
|
+
> \* chrome131 JA3 is **non-deterministic** — it changes every connection (verified across 4 back-to-back probes: 4 distinct JA3 hashes). The table shows one sample; see §4.1.
|
|
57
|
+
|
|
58
|
+
### Constant values across all profiles (empirically confirmed)
|
|
59
|
+
|
|
60
|
+
| Field | Value |
|
|
61
|
+
|---|---|
|
|
62
|
+
| TLS version (negotiated) | 772 (TLS 1.3) |
|
|
63
|
+
| TLS version (record) | 771 |
|
|
64
|
+
| ALPN | `h2`, `http/1.1` |
|
|
65
|
+
| Signature algorithms (Chrome/Edge) | `ecdsa_secp256r1_sha256, rsa_pss_rsae_sha256, rsa_pkcs1_sha256, edsa_secp384r1_sha384, rsa_pss_rsae_sha384, rsa_pkcs1_sha384, rsa_pss_rsae_sha512, rsa_pkcs1_sha512` |
|
|
66
|
+
| Signature algorithms (Firefox) | adds `ecdsa_secp521r1_sha512, ecdsa_sha1, rsa_pkcs1_sha1` |
|
|
67
|
+
| EC point formats | `[0]` (uncompressed) — all profiles |
|
|
68
|
+
| HEADER_TABLE_SIZE | 65536 — all profiles |
|
|
69
|
+
| ENABLE_PUSH | 0 — all profiles |
|
|
70
|
+
| INITIAL_WINDOW_SIZE (Chrome/Edge) | 6291456 |
|
|
71
|
+
| INITIAL_WINDOW_SIZE (Firefox) | 131072 |
|
|
72
|
+
| INITIAL_WINDOW_SIZE (Safari) | 4194304 (v17) / 2097152 (v18+) |
|
|
73
|
+
| compress_certificate algo | `brotli (2)` (Chrome/Edge), `zlib,brotli,zstd` (Firefox), `zlib` (Safari) |
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## 2. Missing Captures
|
|
78
|
+
|
|
79
|
+
### 2.1 Profiles with **no** golden capture yet
|
|
80
|
+
|
|
81
|
+
We currently have golden captures **only** for `chrome-140` and `firefox-128`. The following curl_cffi profiles — spanning Chrome, Firefox, Safari, and Edge — have **no golden capture**:
|
|
82
|
+
|
|
83
|
+
`chrome99–chrome136` (14 targets), `edge99`, `edge101`, `safari155`, `safari170`, `safari180`, `safari184`, `safari180_ios`, `safari260`, `firefox133`, `firefox135`.
|
|
84
|
+
|
|
85
|
+
This means the golden-comparison test (Category 14) only exercises two profiles, and those two are synthetic stubs (§3.1).
|
|
86
|
+
|
|
87
|
+
### 2.2 Do the existing captures match current curl-cffi output?
|
|
88
|
+
|
|
89
|
+
**No.** The existing `chrome-140` and `firefox-128` captures are not representative of any real curl-cffi profile:
|
|
90
|
+
|
|
91
|
+
| Property | Existing golden stub | Real curl-cffi (any profile) |
|
|
92
|
+
|---|---|---|
|
|
93
|
+
| ClientHello size | **96 bytes** | **517–1841 bytes** |
|
|
94
|
+
| Cipher suites | **2** (`0x1301`, `0x1302`) | **16–21** |
|
|
95
|
+
| GREASE | **none** | **always present** (ciphers + extensions) |
|
|
96
|
+
| `client_random` | sequential (`00010203…` / `80818283…`) | cryptographically random |
|
|
97
|
+
| Extensions | **4** (`server_name, supported_groups, ec_point_formats, ALPN`) | **15–18** |
|
|
98
|
+
| JA3 hash | `853b03398669dbeffb6116ecd6e6beb6` (both chrome AND firefox!) | profile-specific (see matrix) |
|
|
99
|
+
| HTTP/2 SETTINGS | 2 settings: `MAX_CONCURRENT_STREAMS=100, INITIAL_WINDOW_SIZE=65536` | browser-family-specific (§3.3) |
|
|
100
|
+
|
|
101
|
+
The two stubs differ **only** in `client_random` (masked), so they produce the **same** JA3 hash — a dead giveaway they are test fixtures, not captures.
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## 3. Divergence Analysis
|
|
106
|
+
|
|
107
|
+
### 3.1 ClientHello (TLS) divergence
|
|
108
|
+
|
|
109
|
+
The most granular fingerprint signal is the **peetprint** string (tls.peet.ws's ordering-aware format). Format: `grease|record_ver|alpn|supported_groups|sig_algs|compression|ciphers|extensions-grease-grease`. A few representative examples:
|
|
110
|
+
|
|
111
|
+
```
|
|
112
|
+
chrome-131: GREASE-772-771|2-1.1|GREASE-4588-29-23-24|1027-2052-1025-1283-2053-1281-2054-1537|1|2|GREASE-4865-4866-...|0-10-11-13-16-17513-18-23-27-35-43-45-5-51-65037-65281-GREASE-GREASE
|
|
113
|
+
firefox-133: 772-771|2-1.1|4588-29-23-24-25-256-257|1027-1283-1539-2052-2053-2054-1025-1281-1537-515-513|1|1-2-3|4865-4867-...|0-10-11-13-16-23-27-28-34-35-43-45-5-51-65037-65281
|
|
114
|
+
safari-17: GREASE-772-771-770-769|2-1.1|GREASE-29-23-24-25|1027-2052-...|1|1|GREASE-4865-...|0-10-11-13-16-18-21-23-27-43-45-5-51-65281-GREASE-GREASE
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
**Key structural divergences by browser family:**
|
|
118
|
+
|
|
119
|
+
| Feature | Chrome/Edge | Firefox | Safari |
|
|
120
|
+
|---|---|---|---|
|
|
121
|
+
| GREASE in supported_versions | yes (leading) | **no** | yes |
|
|
122
|
+
| supported_versions contents | `[GREASE, TLS 1.3, TLS 1.2]` | `[TLS 1.3, TLS 1.2]` | `[GREASE, TLS 1.3, TLS 1.2, TLS 1.1, TLS 1.0]` |
|
|
123
|
+
| supported_groups | `GREASE, 4588, 29, 23, 24` (4588=X25519MLKEM768 PQ) | `4588, 29, 23, 24, 25, 256, 257` (adds P-521 + ffdhe) | `GREASE, 29, 23, 24, 25` |
|
|
124
|
+
| key_share groups (sent) | `GREASE, 4588, 29` | `4588, 29, 23` | `GREASE, 29` |
|
|
125
|
+
| key_share group 4588 | present (Chrome 124+) | present (133+) | **absent** (no PQ) |
|
|
126
|
+
| compress_certificate | brotli only | zlib+brotli+zstd | zlib only |
|
|
127
|
+
| padding extension (21) | Chrome ≤116, Edge | no | **yes** |
|
|
128
|
+
| application_settings | 17513 (old) / 17613 (new) | no | no |
|
|
129
|
+
| delegated_credentials (34) | no | yes | no |
|
|
130
|
+
| record_size_limit (28) | no | yes | no |
|
|
131
|
+
| 3DES ciphers | no | no | **yes** |
|
|
132
|
+
|
|
133
|
+
> **Note on cipher 4588 / 25497:** curl-cffi uses IANA's newer `X25519MLKEM768` (4588) for Chrome 131+ and Firefox 133+; Chrome 124 used the older `X25519Kyber768Draft00` (25497). If `@browsercore` hardcodes the older number for "Chrome 124-era" fingerprints, that is correct — but the modern post-quantum group identifier is 4588.
|
|
134
|
+
|
|
135
|
+
### 3.2 JA3 / JA4 divergence
|
|
136
|
+
|
|
137
|
+
JA3 = MD5 of `version,ciphers,extensions,supported_groups,ec_point_formats`. Because extension **order** and **GREASE** are included, JA3 is extremely sensitive to ordering.
|
|
138
|
+
|
|
139
|
+
- Synthetic stub JA3: `853b03398669dbeffb6116ecd6e6beb6` (no GREASE, 4 extensions).
|
|
140
|
+
- Real chrome-131 JA3 sample: `fb519300321e7e157792ac8d3a77e9ee`.
|
|
141
|
+
- Real firefox-133 JA3: `2d692a4485ca2f5f2b10ecb2d2909ad3`.
|
|
142
|
+
|
|
143
|
+
**JA4** is the modern replacement (format `t{cc}e{aln}_{b}_{c}_{f}`). It is less order-sensitive in the `a`/`b`/`c` parts but the `a` part encodes extension count, cipher count, SNI flag, and TLS version — so extension count matters. Our library's `computeJa4` (verified against §1 matrix) correctly reproduces these; the divergence risk is in the **input** ClientHello, not the computation.
|
|
144
|
+
|
|
145
|
+
### 3.3 HTTP/2 SETTINGS divergence
|
|
146
|
+
|
|
147
|
+
The SETTINGS frame contents are **not randomized** (empty `randomizedFields` in the stub meta.json is correct in principle, but the stub's 2-setting content is wrong). Real values:
|
|
148
|
+
|
|
149
|
+
| Setting (id) | Chrome/Edge | Firefox | Safari 17 | Safari 18+ |
|
|
150
|
+
|---|---|---|---|---|
|
|
151
|
+
| HEADER_TABLE_SIZE (1) | 65536 | 65536 | **absent** | **absent** |
|
|
152
|
+
| ENABLE_PUSH (2) | 0 | 0 | 0 | 0 |
|
|
153
|
+
| MAX_CONCURRENT_STREAMS (3) | *absent†* / 1000‡ | absent | 100 | 100 |
|
|
154
|
+
| INITIAL_WINDOW_SIZE (4) | 6291456 | 131072 | 4194304 | 2097152 |
|
|
155
|
+
| MAX_FRAME_SIZE (5) | absent | 16384 | absent | absent |
|
|
156
|
+
| MAX_HEADER_LIST_SIZE (6) | 262144 | absent | absent | absent |
|
|
157
|
+
| ENABLE_CONNECT_PROTOCOL (8) | absent | absent | absent | 1 |
|
|
158
|
+
| NO_RFC7540_PRIORITIES (9) | absent | absent | absent | 1 |
|
|
159
|
+
|
|
160
|
+
> † Chrome 119+ omits `MAX_CONCURRENT_STREAMS`; Chrome 116 and earlier include it (value 1000). Edge always includes it (1000).
|
|
161
|
+
>
|
|
162
|
+
> ‡ Edge uniquely sends `MAX_CONCURRENT_STREAMS=1000` together with `HEADER_TABLE_SIZE` — a combination not seen in Chrome itself.
|
|
163
|
+
|
|
164
|
+
**SETTINGS order matters** for the Akamai fingerprint (the `m,a,s,p` suffix in the akamai fingerprint string encodes header-table, window, concurrent-streams, push order). Real Chrome-131 akamai fingerprint: `1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p`.
|
|
165
|
+
|
|
166
|
+
### 3.4 Request header ordering divergence
|
|
167
|
+
|
|
168
|
+
**Pseudo-header order** (this is a strong fingerprint signal):
|
|
169
|
+
|
|
170
|
+
| Browser | Pseudo-header order |
|
|
171
|
+
|---|---|
|
|
172
|
+
| Chrome / Edge | `:method` `:authority` `:scheme` `:path` |
|
|
173
|
+
| Firefox | `:method` `:path` `:authority` `:scheme` |
|
|
174
|
+
| Safari | `:method` `:scheme` `:path` `:authority` |
|
|
175
|
+
|
|
176
|
+
**Request header order** (regular headers after pseudo-headers), empirically observed:
|
|
177
|
+
|
|
178
|
+
Chrome 131:
|
|
179
|
+
```
|
|
180
|
+
:method :authority :scheme :path
|
|
181
|
+
sec-ch-ua | sec-ch-ua-mobile | sec-ch-ua-platform | upgrade-insecure-requests | user-agent |
|
|
182
|
+
accept | sec-fetch-site | sec-fetch-mode | sec-fetch-user | sec-fetch-dest |
|
|
183
|
+
accept-encoding | accept-language | priority
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
Firefox 133:
|
|
187
|
+
```
|
|
188
|
+
:method :path :authority :scheme
|
|
189
|
+
user-agent | accept | accept-language | accept-encoding | upgrade-insecure-requests |
|
|
190
|
+
sec-fetch-dest | sec-fetch-mode | sec-fetch-site | sec-fetch-user | priority | te: trailers
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Safari 180:
|
|
194
|
+
```
|
|
195
|
+
:method :scheme :authority :path
|
|
196
|
+
sec-fetch-dest | user-agent | accept | sec-fetch-site | sec-fetch-mode |
|
|
197
|
+
accept-language | priority | accept-encoding
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
**Divergence notes:**
|
|
201
|
+
- Firefox places `:path` **second** (before `:authority/:scheme`) and puts `user-agent` first among regular headers.
|
|
202
|
+
- Safari places `:scheme` **second** and `:authority` before `:path`; does **not** include `accept-encoding` adjacent to `accept` — it's separated by `priority` and sits last.
|
|
203
|
+
- Only Firefox sends `te: trailers`.
|
|
204
|
+
- Only Chrome/Safari send `priority: u=0, i` as a request header (not just the HEADERS-frame priority).
|
|
205
|
+
|
|
206
|
+
### 3.5 HEADERS frame priority / framing divergence
|
|
207
|
+
|
|
208
|
+
The priority field inside the HEADERS frame (the 5-byte priority block) differs:
|
|
209
|
+
|
|
210
|
+
| Browser | weight | depends_on | exclusive |
|
|
211
|
+
|---|---|---|---|
|
|
212
|
+
| Chrome / Edge | 256 | 0 | **1** |
|
|
213
|
+
| Firefox | 256 | 0 | **0** |
|
|
214
|
+
| Safari 17 | **255** | 0 | 0 |
|
|
215
|
+
| Safari 18+ | 256 | 0 | 0 |
|
|
216
|
+
|
|
217
|
+
Chrome's `exclusive=1` vs others' `0` is a clean distinguisher. Safari 17's `weight=255` (vs 256) is another.
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
## 4. Specific Behavior Gaps
|
|
222
|
+
|
|
223
|
+
These are behaviors curl-impersonate exhibits that `@browsercore` likely does **not** yet replicate.
|
|
224
|
+
|
|
225
|
+
### 4.1 Non-deterministic extension ordering (Chrome)
|
|
226
|
+
|
|
227
|
+
**Empirically verified:** probing chrome131 four times back-to-back produced **four distinct JA3 hashes**:
|
|
228
|
+
|
|
229
|
+
```
|
|
230
|
+
probe 0: 19f1a330145357c27fc7a42356343a1e
|
|
231
|
+
probe 1: 39a71348904055386f5d97dda94ab014
|
|
232
|
+
probe 2: 39ded8f8ff9b0b36a4e2f8524c3a014d
|
|
233
|
+
probe 3: b9d2b13f886101bb260606bd53cc9731
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
Each probe had a **completely different extension order** (the entire block is permuted, not just GREASE values). This is curl-impersonate behavior — it shuffles the Chrome extension order to mimic real Chrome's variability. **Implication for `@browsercore`:** golden comparison for Chrome profiles cannot be a single fixed byte string; it must either mask the extension block (as our new `chrome-131` meta.json does) or compare a canonicalized/ordered form. The current `chrome-140` stub does not capture this at all.
|
|
237
|
+
|
|
238
|
+
### 4.2 GREASE value diversity
|
|
239
|
+
|
|
240
|
+
GREASE occupies both the cipher-slot and extension-slot positions **and** specific values inside `supported_versions`, `key_share`, and `supported_groups`. The oracle normalizes these away for JA3 display, but on the wire the specific 0x?A?A values vary. Our stubs have zero GREASE — a real fingerprinting service would flag this instantly.
|
|
241
|
+
|
|
242
|
+
### 4.3 Frame padding
|
|
243
|
+
|
|
244
|
+
Safari and Edge include the TLS `padding` extension (21) to fix ClientHello size — Safari's captured ClientHello is exactly 517 bytes; without padding it would be shorter. `@browsercore` would need profile-driven padding.
|
|
245
|
+
|
|
246
|
+
### 4.4 Post-quantum key share (X25519MLKEM768)
|
|
247
|
+
|
|
248
|
+
Chrome 124+ and Firefox 133+ advertise the hybrid PQ group `4588` (X25519MLKEM768) in both `supported_groups` and `key_share`. The group number (4588 vs the older draft 25497) is a version signal. `@browsercore` profiles for modern Chrome/Firefox must include this or be flagged as outdated.
|
|
249
|
+
|
|
250
|
+
### 4.5 SETTINGS ACK timing / frame sequence
|
|
251
|
+
|
|
252
|
+
The oracle shows the **first** client frames are always `SETTINGS` then `WINDOW_UPDATE`, then `HEADERS`. curl-impersonate sends its SETTINGS immediately on connection. If `@browsercore` delays SETTINGS, batches it differently, or responds to the server's SETTINGS before sending its own, that ordering is fingerprintable. The peetprint `akamai_fingerprint_hash` captures the SETTINGS values; a timing-based oracle would capture ordering.
|
|
253
|
+
|
|
254
|
+
### 4.6 application_settings (GREASE-adjacent) identifier
|
|
255
|
+
|
|
256
|
+
Chrome sends `application_settings` — but the IANA identifier changed: older curl-cffi Chrome uses **17513** (`application_settings_old`), newer (chrome-133+) uses **17613**. This is a version signal. `@browsercore` must pick the right id per profile.
|
|
257
|
+
|
|
258
|
+
### 4.7 ECH (Encrypted ClientHello) extension (0xfe0d)
|
|
259
|
+
|
|
260
|
+
All profiles send `extensionEncryptedClientHello` (boringssl) with a large random payload (~280 bytes). The payload is randomized and must be masked. Our stubs omit ECH entirely. (Note: the ECH extension is present but the inner is usually to a no-op config for fingerprinting — the bytes still matter.)
|
|
261
|
+
|
|
262
|
+
---
|
|
263
|
+
|
|
264
|
+
## 5. Recommended New Test Cases
|
|
265
|
+
|
|
266
|
+
Concrete capture scenarios to add (all are curl_cffi-available targets):
|
|
267
|
+
|
|
268
|
+
| # | Test case | Rationale |
|
|
269
|
+
|---|---|---|
|
|
270
|
+
| 1 | **chrome-131 / chrome-133 / chrome-136 golden** (non-deterministic) | Modern Chrome with PQ key share (4588), `17613`, brotli. Must mask extension block in comparison due to §4.1. |
|
|
271
|
+
| 2 | **firefox-133 / firefox-135 golden** | Firefox with 17-cipher suite, delegated_credentials, record_size_limit, zlib+brotli+zstd. |
|
|
272
|
+
| 3 | **safari-17 / safari-18 golden** | Safari with padding extension, weight=255 (v17), `NO_RFC7540_PRIORITIES`, `ENABLE_CONNECT_PROTOCOL`. |
|
|
273
|
+
| 4 | **edge-99 / edge-101 golden** | Edge-specific: `MAX_CONCURRENT_STREAMS=1000` + HEADER_TABLE_SIZE together, `padding`, `17513`. |
|
|
274
|
+
| 5 | **PEETPRINT comparison test** | Extend golden comparison to the full `peetprint` string (ordering-aware) rather than raw bytes alone. The oracle exposes it for every connection. |
|
|
275
|
+
| 6 | **JA3/JA4 conformance test** | Assert our `computeJa3`/`computeJa4` outputs match the oracle's reported `ja3_hash`/`ja4` for each profile — validates the parsing path independently of byte generation. |
|
|
276
|
+
| 7 | **HTTP/2 SETTINGS frame test** | Assert the first SETTINGS frame bytes (reconstructable from the profile) match the oracle's `sent_frames[SETTINGS]` for each browser family. |
|
|
277
|
+
| 8 | **Pseudo-header ordering test** | Assert `:method/:scheme/:authority/:path` order matches browser family (Chrome vs Firefox vs Safari differ). |
|
|
278
|
+
| 9 | **Priority framing test** | Assert HEADERS-frame priority `weight/depends_on/exclusive` matches family (Chrome `exclusive=1`, others `0`; Safari v17 `weight=255`). |
|
|
279
|
+
| 10 | **GREASE presence test** | Assert GREASE values appear in ciphers, extensions, supported_versions, key_share, supported_groups for Chrome/Edge/Safari; assert Firefox has none. |
|
|
280
|
+
|
|
281
|
+
---
|
|
282
|
+
|
|
283
|
+
## 6. New Golden Captures Created
|
|
284
|
+
|
|
285
|
+
Three new profiles were captured from real curl_cffi connections (via transparent SOCKS5 relay to `tls.peet.ws` with correct SNI) and saved in the existing `captures/<profile>/<protocol>/<record>.{bin,meta.json}` layout:
|
|
286
|
+
|
|
287
|
+
| Profile | client_hello.bin | settings.bin | Validation |
|
|
288
|
+
|---|---|---|---|
|
|
289
|
+
| `chrome-131` | 1753 bytes | 33 bytes (4 settings) | Real curl_cffi bytes; JA3 non-deterministic (see §4.1); extension block masked as `grease` |
|
|
290
|
+
| `firefox-133` | 1797 bytes | 33 bytes (4 settings) | **Byte-perfect JA3 match** to oracle (`2d692a4485ca2f5f2b10ecb2d2909ad3`) — confirms capture authenticity |
|
|
291
|
+
| `safari-17` | 517 bytes | 27 bytes (3 settings) | GREASE-stripped JA3 matches oracle (`773906b0efdefa24a7f2b8eb6985bf37`) |
|
|
292
|
+
|
|
293
|
+
Each directory also contains:
|
|
294
|
+
- `tls/client_hello.meta.json` — schema-valid sidecar (source `curl-impersonate`, randomized fields listed).
|
|
295
|
+
- `http2/settings.meta.json` — `randomizedFields: []` (SETTINGS are deterministic).
|
|
296
|
+
- `oracle_capture.reference.json` — the full oracle JSON response for that capture (JA3, JA4, peetprint, extension list, SETTINGS, headers, priority).
|
|
297
|
+
|
|
298
|
+
**Files live in:**
|
|
299
|
+
- `captures/chrome-131/`
|
|
300
|
+
- `captures/firefox-133/`
|
|
301
|
+
- `captures/safari-17/`
|
|
302
|
+
|
|
303
|
+
### Capture methodology
|
|
304
|
+
A local SOCKS5 proxy relayed curl_cffi's TCP connection to the real `tls.peet.ws:443`, reading (and saving) the first TLS flight (the ClientHello) before relaying it upstream. Because curl_cffi believes it is talking to `tls.peet.ws`, SNI and all extensions are authentic. firefox-133's captured bytes reproduce the oracle's JA3 hash **exactly**, proving the relay does not alter bytes.
|
|
305
|
+
|
|
306
|
+
### Notes on the randomized-field masks
|
|
307
|
+
- `client_hello.meta.json` masks `client_random` (offset 12, length 32, reason `random`) and the **entire extension block** (reason `grease`). This is correct for Chrome (non-deterministic order) and conservative-but-valid for Firefox/Safari (deterministic order, but GREASE values still vary per RFC 8701).
|
|
308
|
+
- The synthetic stubs' mask claiming `ephemeral_key` at offset 49 is **meaningless** for the stubs (no key_share extension exists at that offset) — any new comparison should derive masks from the actual captured bytes.
|
|
309
|
+
|
|
310
|
+
---
|
|
311
|
+
|
|
312
|
+
## Appendix A — Environment & Reproduction
|
|
313
|
+
|
|
314
|
+
- **curl_cffi** 0.13.0, Python 3.9.20 (macOS).
|
|
315
|
+
- **Oracle:** `https://tls.peet.ws/api/all` (returns JSON). Reachable throughout the session.
|
|
316
|
+
- **Probe scripts:** `testing/captures/_probe/probe.py` (profile sweep), `analyze.py` (detail), `socks_capture.py` (transparent SOCKS5 capture), `make_golden.py` (golden file generation), `validate_golden.py` (validation). All per-profile oracle JSON is in `testing/captures/_probe/output/<profile>.json`.
|
|
317
|
+
- **Library versions tested against:** `@browsercore/testing` golden loader (`parseCaptureMeta` in `src/golden/golden.ts`) and `computeJa3`/`computeJa4` (`src/fingerprint/ja3.ts`, `ja4.ts`) — confirmed to accept the new meta.json schema.
|
|
318
|
+
|
|
319
|
+
## Appendix B — Anti-hallucination log
|
|
320
|
+
|
|
321
|
+
- Every JA3/JA4/peetprint/SETTINGS/header value is pasted from actual oracle JSON output. No values were fabricated.
|
|
322
|
+
- The chrome-131 non-determinism (§4.1) and firefox-133 byte-perfect match are **empirically verified** (repeated probes + JA3 cross-check).
|
|
323
|
+
- safari-17 JA3 matches only **after GREASE-stripping** — the oracle strips GREASE before computing its displayed JA3; the raw captured bytes include GREASE. This is labeled, not hidden.
|
|
324
|
+
- The "ephemeral_key" mask in the existing stubs is flagged as *meaningless* because the stubs contain no ECDHE key share — this is a claim about the stub content, verified by hex inspection.
|
|
325
|
+
- **Not verified / limits:** (1) We did not compare `@browsercore`'s actual emitted bytes against these captures — this report identifies *where* divergence is likely, not a byte-level diff of library output. (2) curl_cffi's chrome extension-ordering behavior (§4.1) is observed but its cause (curl-impersonate internals) is *[inferred]*. (3) Captures were taken at a point in time; curl-impersonate updates may shift fingerprints (the report documents the curl_cffi 0.13.0 baseline).
|