ja4proxy 0.1.0__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/PKG-INFO +691 -688
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/README.md +6 -3
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy/__init__.py +1 -1
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy/certs.py +24 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy/cli.py +8 -1
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy/engine/bridge.py +25 -18
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy/parser.py +15 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy/profiles.py +127 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy/server.py +344 -195
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy.egg-info/PKG-INFO +691 -688
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/pyproject.toml +1 -1
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/setup.cfg +4 -4
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/tests/test_live_proxy.py +100 -18
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/LICENSE +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy/engine/loader.py +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy.egg-info/SOURCES.txt +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy.egg-info/dependency_links.txt +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy.egg-info/entry_points.txt +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy.egg-info/requires.txt +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/ja4proxy.egg-info/top_level.txt +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/tests/test_certs.py +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/tests/test_engine.py +0 -0
- {ja4proxy-0.1.0 → ja4proxy-0.2.0}/tests/test_parser.py +0 -0
|
@@ -1,688 +1,691 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: ja4proxy
|
|
3
|
-
Version: 0.
|
|
4
|
-
Summary: High-performance universal proxy server with browser TLS (JA4) and HTTP/2 impersonation
|
|
5
|
-
Author-email: cipherwire <dev@cipherwire.link>
|
|
6
|
-
License-Expression: MIT
|
|
7
|
-
Requires-Python: >=3.10
|
|
8
|
-
Description-Content-Type: text/markdown
|
|
9
|
-
License-File: LICENSE
|
|
10
|
-
Requires-Dist: cryptography>=41.0.0
|
|
11
|
-
Requires-Dist: click>=8.0.0
|
|
12
|
-
Dynamic: license-file
|
|
13
|
-
|
|
14
|
-
<div align="center">
|
|
15
|
-
|
|
16
|
-
```text
|
|
17
|
-
_ __ __
|
|
18
|
-
(_)___ _/ // / ____ _________ _ ____ __
|
|
19
|
-
/ / __ `/ // /_/ __ \/ ___/ __ \| |/_/ / / /
|
|
20
|
-
/ / /_/ /__ __/ /_/ / / / /_/ /> </ /_/ /
|
|
21
|
-
__/ /\__,_/ /_/ / .___/_/ \____/_/|_|\__, /
|
|
22
|
-
/___/ /_/ /____/
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
### High-performance universal forward proxy with authentic browser TLS (JA4/JA3) and HTTP/2 framing
|
|
26
|
-
|
|
27
|
-
<p>
|
|
28
|
-
<img src="https://img.shields.io/badge/platform-Windows%20%7C%20Linux%20%7C%20macOS-2563eb" alt="Platforms">
|
|
29
|
-
<img src="https://img.shields.io/badge/engine-uTLS%20%2B%20fhttp-4285F4" alt="Engine">
|
|
30
|
-
<img src="https://img.shields.io/badge/protocols-HTTP%20%7C%20HTTPS%20%7C%20SOCKS5-00599C" alt="Protocols">
|
|
31
|
-
<img src="https://img.shields.io/badge/fingerprint-JA4%20%2F%20JA3%20%2F%20Akamai%20H2-6366f1" alt="Fingerprint">
|
|
32
|
-
<img src="https://img.shields.io/badge/license-MIT-green" alt="License: MIT">
|
|
33
|
-
<img src="https://img.shields.io/badge/status-production-22c55e" alt="Status">
|
|
34
|
-
</p>
|
|
35
|
-
|
|
36
|
-
</div>
|
|
37
|
-
|
|
38
|
-
---
|
|
39
|
-
|
|
40
|
-
## Table of Contents
|
|
41
|
-
|
|
42
|
-
- [The Fingerprinting Problem](#the-fingerprinting-problem)
|
|
43
|
-
- [How ja4proxy Works](#how-ja4proxy-works)
|
|
44
|
-
- [Architecture & Protocol Pipeline](#architecture--protocol-pipeline)
|
|
45
|
-
- [Comparative Analysis](#comparative-analysis)
|
|
46
|
-
- [Key Specifications](#key-specifications)
|
|
47
|
-
- [Installation](#installation)
|
|
48
|
-
- [Quick Start](#quick-start)
|
|
49
|
-
- [Multi-Language Client Integration](#multi-language-client-integration)
|
|
50
|
-
- [Python (requests, httpx, aiohttp)](#python-requests-httpx-aiohttp)
|
|
51
|
-
- [Node.js (axios, undici, fetch)](#nodejs-axios-undici-fetch)
|
|
52
|
-
- [Go (net/http)](#go-nethttp)
|
|
53
|
-
- [Rust (reqwest)](#rust-reqwest)
|
|
54
|
-
- [Scrapy Framework](#scrapy-framework)
|
|
55
|
-
- [Dynamic Per-Request Directives](#dynamic-per-request-directives)
|
|
56
|
-
- [Via Proxy Credentials (Tag Routing)](#via-proxy-credentials-tag-routing)
|
|
57
|
-
- [Via Custom Request Headers](#via-custom-request-headers)
|
|
58
|
-
- [Residential Proxy Chaining](#residential-proxy-chaining)
|
|
59
|
-
- [Supported Browser Presets](#supported-browser-presets)
|
|
60
|
-
- [Production Deployment](#production-deployment)
|
|
61
|
-
- [Docker & Docker Compose](#docker--docker-compose)
|
|
62
|
-
- [Systemd Service Unit](#systemd-service-unit)
|
|
63
|
-
- [System-Wide CA Trust Installation](#system-wide-ca-trust-installation)
|
|
64
|
-
- [Security & Engineering Boundaries](#security--engineering-boundaries)
|
|
65
|
-
- [CLI Reference](#cli-reference)
|
|
66
|
-
- [Automated Verification Suite](#automated-verification-suite)
|
|
67
|
-
- [License](#license)
|
|
68
|
-
|
|
69
|
-
---
|
|
70
|
-
|
|
71
|
-
## The Fingerprinting Problem
|
|
72
|
-
|
|
73
|
-
Modern anti-bot mitigation platforms (Cloudflare Turnstile / Bot Management, Akamai Edge Workers, Imperva, Datadome, Kasada) no longer rely solely on IP reputation. Instead, automated scrapers and bots are blocked at Layer 7 by inspecting client-side protocol artifacts during the connection handshake:
|
|
74
|
-
|
|
75
|
-
```text
|
|
76
|
-
Layer 7 Defense Hierarchy
|
|
77
|
-
├── 1. TLS ClientHello Fingerprinting (JA3 / JA4)
|
|
78
|
-
│ ├── Cipher suites order & availability
|
|
79
|
-
│ ├── Extension list & GREASE (RFC 8701) injection
|
|
80
|
-
│ ├── Elliptic curve groups & signature algorithms
|
|
81
|
-
│ └── ALPN (Application-Layer Protocol Negotiation)
|
|
82
|
-
├── 2. HTTP/2 Transport Fingerprinting (Akamai HTTP2 Hash)
|
|
83
|
-
│ ├── SETTINGS frame parameters (HEADER_TABLE_SIZE, MAX_CONCURRENT_STREAMS)
|
|
84
|
-
│ ├── WINDOW_UPDATE increments
|
|
85
|
-
│ ├── Pseudo-header serialization order (:method, :authority, :scheme, :path)
|
|
86
|
-
│ └── Priority trees and frame dependency graphs
|
|
87
|
-
└── 3. Layer 7 HTTP Header Consistency
|
|
88
|
-
├── Sec-CH-UA, Sec-CH-UA-Mobile, Sec-CH-UA-Platform
|
|
89
|
-
└── User-Agent string alignment with actual cryptographic capabilities
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
Standard networking stacks (OpenSSL in Python `urllib3`, Go `crypto/tls`, Node.js `tls`) expose hard-coded, static handshakes that immediately flag requests as non-browser traffic.
|
|
93
|
-
|
|
94
|
-
### Fingerprint Breakdown: OpenSSL vs. ja4proxy (Chrome 133)
|
|
95
|
-
|
|
96
|
-
| Component | Standard Python / Node / Go | ja4proxy (Chrome 133 Profile) |
|
|
97
|
-
| :--- | :--- | :--- |
|
|
98
|
-
| **JA4 Signature** | `t13d030800_...` (Flagged as Bot) | `t13d1516h2_8daaf6152771_d8a2da3f94cd` (Identical to Chrome 133) |
|
|
99
|
-
| **GREASE Support** | None (RFC 8701 absent) | Fully randomized GREASE cipher and extension values |
|
|
100
|
-
| **HTTP Version** | Often forced to HTTP/1.1 | Native HTTP/2 (RFC 7540) with authentic browser stream frames |
|
|
101
|
-
| **Pseudo-Header Order** | Variable or alphabetical | `:method`, `:authority`, `:scheme`, `:path` |
|
|
102
|
-
| **H2 SETTINGS** | Default OpenSSL/Go parameters | `1:65536, 2:0, 4:6291456, 6:262144` matching Chromium |
|
|
103
|
-
|
|
104
|
-
---
|
|
105
|
-
|
|
106
|
-
## How ja4proxy Works
|
|
107
|
-
|
|
108
|
-
`ja4proxy` is an in-process, high-concurrency forward proxy. Rather than forwarding raw TCP sockets (which passes through your runtime's detectable OpenSSL handshake), `ja4proxy` terminates client requests locally and re-originates the upstream connection using an in-process uTLS and fhttp engine written in Go.
|
|
109
|
-
|
|
110
|
-
```text
|
|
111
|
-
Any Existing Client Application
|
|
112
|
-
(cURL / requests / Scrapy / Playwright / Node / Go / Rust)
|
|
113
|
-
│
|
|
114
|
-
HTTP CONNECT / Plain HTTP / SOCKS5 binary
|
|
115
|
-
│
|
|
116
|
-
▼
|
|
117
|
-
┌────────────────────────────────────────────────────────────────────────┐
|
|
118
|
-
│ ja4proxy Runtime (127.0.0.1:8888) │
|
|
119
|
-
│ │
|
|
120
|
-
│ [Ingress Sniffer] -> Auto-detects protocol by magic byte: │
|
|
121
|
-
│ - 0x05 (SOCKS5 RFC 1928) │
|
|
122
|
-
│ - CONNECT host:443 (HTTPS Tunnel) │
|
|
123
|
-
│ - GET / POST / http://... (HTTP Gateway) │
|
|
124
|
-
│ │
|
|
125
|
-
│ [Directive Parser] -> Extracts profile, session, upstream proxy: │
|
|
126
|
-
│ - Via credential tags: user-profile_chrome133 │
|
|
127
|
-
│ - Via custom headers: X-JA4-Profile │
|
|
128
|
-
│ │
|
|
129
|
-
│ [SSRF Guard] -> Validates target IP against private/loopback CIDRs │
|
|
130
|
-
│ │
|
|
131
|
-
│ [TLS Interceptor] -> Generates dynamic in-memory SAN leaf certs │
|
|
132
|
-
│ │
|
|
133
|
-
│ [Native Zero-GIL FFI] -> Dispatches request into Go uTLS engine │
|
|
134
|
-
└───────────────────────────────────┬────────────────────────────────────┘
|
|
135
|
-
│
|
|
136
|
-
Outbound: Real Chrome 133 JA4 + HTTP/2 Multiplexing
|
|
137
|
-
│
|
|
138
|
-
▼
|
|
139
|
-
Upstream Target (WAF / Protected Host)
|
|
140
|
-
```
|
|
141
|
-
|
|
142
|
-
---
|
|
143
|
-
|
|
144
|
-
## Architecture & Protocol Pipeline
|
|
145
|
-
|
|
146
|
-
### 1. Single-Port Protocol Sniffing
|
|
147
|
-
`ja4proxy` binds to a single port and inspects the initial byte stream using non-destructive socket inspection:
|
|
148
|
-
* Byte `0x05`: Dispatches to the binary SOCKS5 state machine (RFC 1928), negotiating `NO AUTH` or `USERNAME/PASSWORD` before taking target host and port.
|
|
149
|
-
* ASCII `CONNECT`: Intercepts HTTPS tunnels, performs an SSL handshake using dynamic per-hostname leaf certificates signed on-the-fly by the local CA, and decrypts request bytes.
|
|
150
|
-
* ASCII `GET / POST / HEAD`: Handles standard forward proxy requests or gateway URLs (`http://127.0.0.1:8888/https://example.com`).
|
|
151
|
-
|
|
152
|
-
### 2. Zero-GIL Native C-Shared FFI
|
|
153
|
-
Python manages client I/O asynchronously via `asyncio`. When dispatching outbound HTTP/TLS requests, `ja4proxy` calls into `libutls` through `ctypes`. The Python Global Interpreter Lock (GIL) is released during socket creation, cryptographic handshakes, and body streaming.
|
|
154
|
-
|
|
155
|
-
### 3. Memory Hygiene & Free Routine
|
|
156
|
-
Native C allocations returned by the Go runtime are wrapped in a structured memory protocol. Once response headers and body buffers are consumed in Python, `freeMemory` is explicitly invoked over the C pointer, preventing memory leaks across long-running proxy processes.
|
|
157
|
-
|
|
158
|
-
---
|
|
159
|
-
|
|
160
|
-
## Comparative Analysis
|
|
161
|
-
|
|
162
|
-
| Feature | ja4proxy | curl-impersonate | tls-client (Python) | Puppeteer / Playwright | Squid / Privoxy |
|
|
163
|
-
| :--- | :--- | :--- | :--- | :--- | :--- |
|
|
164
|
-
| **Drop-in for Any Language** | Yes (via standard proxy env) | CLI / C only | Python only | Requires browser process | Yes |
|
|
165
|
-
| **Zero Code Changes** | Yes (`HTTP_PROXY=...`) | No (must replace curl binary) | No (rewrite to custom client) | No (heavy refactor) | Yes |
|
|
166
|
-
| **JA4 / JA3 Fingerprint** | Authentic Browser Preset | Authentic Browser Preset | Authentic Browser Preset | Authentic Browser | Leaks OS OpenSSL |
|
|
167
|
-
| **HTTP/2 SETTINGS Impersonation** | Yes | Yes | Yes | Yes | No |
|
|
168
|
-
| **RAM per Worker** | ~18 MB | ~15 MB | ~20 MB | ~150 - 300 MB | ~30 MB |
|
|
169
|
-
| **SOCKS5 Ingress** | Yes (Auto-detected) | No | No | No | Separate port |
|
|
170
|
-
| **Upstream Proxy Chaining** | Dynamic per-request | Static flags | Per-client | Per-browser context | Configuration file |
|
|
171
|
-
|
|
172
|
-
---
|
|
173
|
-
|
|
174
|
-
## Key Specifications
|
|
175
|
-
|
|
176
|
-
- **Listen Protocols**: HTTP Gateway, HTTP CONNECT, SOCKS5 (RFC 1928) on a single port.
|
|
177
|
-
- **TLS Engine**: uTLS with randomized GREASE (RFC 8701) and custom extension order permutations.
|
|
178
|
-
- **HTTP Engine**: fhttp with browser-matched HTTP/2 SETTINGS framing and pseudo-header ordering.
|
|
179
|
-
- **Certificate Authority**: 2048-bit RSA Root CA (`ca.pem`) with automatic in-memory leaf certificate generation and Subject Alternative Name (SAN) caching.
|
|
180
|
-
- **Security Boundary**: Pre-flight connection-time SSRF blocking for RFC 1918 private subnets, loopback, and cloud metadata (`169.254.169.254`).
|
|
181
|
-
- **Decompression Sync**: Automatic header reconciliation (`Content-Encoding`, `Content-Length`) to prevent client decoder desync.
|
|
182
|
-
|
|
183
|
-
---
|
|
184
|
-
|
|
185
|
-
## Installation
|
|
186
|
-
|
|
187
|
-
### Via Source
|
|
188
|
-
|
|
189
|
-
```bash
|
|
190
|
-
git clone https://github.com/cipherwire/ja4proxy.git
|
|
191
|
-
cd ja4proxy
|
|
192
|
-
pip install .
|
|
193
|
-
```
|
|
194
|
-
|
|
195
|
-
### Via Docker
|
|
196
|
-
|
|
197
|
-
```bash
|
|
198
|
-
docker build -t ja4proxy .
|
|
199
|
-
docker run -d -p 8888:8888 --name ja4proxy ja4proxy
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
---
|
|
203
|
-
|
|
204
|
-
## Quick Start
|
|
205
|
-
|
|
206
|
-
### 1. Launch the Server
|
|
207
|
-
|
|
208
|
-
```bash
|
|
209
|
-
ja4proxy start --host 127.0.0.1 --port 8888 --profile chrome_133
|
|
210
|
-
```
|
|
211
|
-
|
|
212
|
-
Output:
|
|
213
|
-
```text
|
|
214
|
-
[*] ja4proxy v0.1.0 starting on http://127.0.0.1:8888
|
|
215
|
-
[*] Default Profile: chrome_133
|
|
216
|
-
[*] Protocols Supported: HTTP CONNECT, HTTP Gateway, SOCKS5 (Auto-detect)
|
|
217
|
-
[*] CA Certificate: C:\Users\AnonNp\.ja4proxy\ca.pem
|
|
218
|
-
[*] Ready for HTTP and SOCKS5 clients
|
|
219
|
-
```
|
|
220
|
-
|
|
221
|
-
### 2. Export the Root CA
|
|
222
|
-
|
|
223
|
-
Export the root certificate so your HTTP client can verify intercept certificates without disabling SSL verification:
|
|
224
|
-
|
|
225
|
-
```bash
|
|
226
|
-
ja4proxy ca --export ./ca.pem
|
|
227
|
-
```
|
|
228
|
-
|
|
229
|
-
### 3. Verify TLS Handshake Against tls.peet.ws
|
|
230
|
-
|
|
231
|
-
#### Using HTTP CONNECT:
|
|
232
|
-
```bash
|
|
233
|
-
curl -x http://127.0.0.1:8888 --cacert ./ca.pem https://tls.peet.ws/api/all
|
|
234
|
-
```
|
|
235
|
-
|
|
236
|
-
#### Using SOCKS5 (Identical Port):
|
|
237
|
-
```bash
|
|
238
|
-
curl -x socks5h://127.0.0.1:8888 --cacert ./ca.pem https://tls.peet.ws/api/all
|
|
239
|
-
```
|
|
240
|
-
|
|
241
|
-
#### Using Gateway Mode (No CA certificate required):
|
|
242
|
-
```bash
|
|
243
|
-
curl http://127.0.0.1:8888/https://tls.peet.ws/api/all
|
|
244
|
-
```
|
|
245
|
-
|
|
246
|
-
Verified output from `tls.peet.ws`:
|
|
247
|
-
```json
|
|
248
|
-
{
|
|
249
|
-
"http_version": "h2",
|
|
250
|
-
"tls": {
|
|
251
|
-
"ja4": "t13d1516h2_8daaf6152771_d8a2da3f94cd",
|
|
252
|
-
"ja3": "771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-21,29-23-24,0"
|
|
253
|
-
}
|
|
254
|
-
}
|
|
255
|
-
```
|
|
256
|
-
|
|
257
|
-
---
|
|
258
|
-
|
|
259
|
-
## Multi-Language Client Integration
|
|
260
|
-
|
|
261
|
-
### Python (requests, httpx, aiohttp)
|
|
262
|
-
|
|
263
|
-
#### `requests`
|
|
264
|
-
```python
|
|
265
|
-
import os
|
|
266
|
-
import requests
|
|
267
|
-
|
|
268
|
-
ca_cert = os.path.expanduser("~/.ja4proxy/ca.pem")
|
|
269
|
-
proxies = {
|
|
270
|
-
"http": "http://127.0.0.1:8888",
|
|
271
|
-
"https": "http://127.0.0.1:8888",
|
|
272
|
-
}
|
|
273
|
-
|
|
274
|
-
# Standard requests call immediately adopts Chrome 133 JA4 fingerprint
|
|
275
|
-
res = requests.get("https://tls.peet.ws/api/all", proxies=proxies, verify=ca_cert)
|
|
276
|
-
print("JA4:", res.json()["tls"]["ja4"])
|
|
277
|
-
```
|
|
278
|
-
|
|
279
|
-
#### `httpx` (Async with SOCKS5)
|
|
280
|
-
```python
|
|
281
|
-
import asyncio
|
|
282
|
-
import os
|
|
283
|
-
import httpx
|
|
284
|
-
|
|
285
|
-
async def main():
|
|
286
|
-
ca_cert = os.path.expanduser("~/.ja4proxy/ca.pem")
|
|
287
|
-
async with httpx.AsyncClient(
|
|
288
|
-
proxy="socks5://127.0.0.1:8888",
|
|
289
|
-
verify=ca_cert,
|
|
290
|
-
) as client:
|
|
291
|
-
res = await client.get("https://tls.peet.ws/api/all")
|
|
292
|
-
print("JA4:", res.json()["tls"]["ja4"])
|
|
293
|
-
|
|
294
|
-
asyncio.run(main())
|
|
295
|
-
```
|
|
296
|
-
|
|
297
|
-
---
|
|
298
|
-
|
|
299
|
-
### Node.js (axios, undici, fetch)
|
|
300
|
-
|
|
301
|
-
#### `axios`
|
|
302
|
-
```javascript
|
|
303
|
-
const axios = require('axios');
|
|
304
|
-
const fs = require('fs');
|
|
305
|
-
const https = require('https');
|
|
306
|
-
|
|
307
|
-
const ca = fs.readFileSync('./ca.pem');
|
|
308
|
-
const httpsAgent = new https.Agent({ ca });
|
|
309
|
-
|
|
310
|
-
axios.get('https://tls.peet.ws/api/all', {
|
|
311
|
-
proxy: { host: '127.0.0.1', port: 8888 },
|
|
312
|
-
httpsAgent: httpsAgent
|
|
313
|
-
}).then(res => {
|
|
314
|
-
console.log('JA4:', res.data.tls.ja4);
|
|
315
|
-
});
|
|
316
|
-
```
|
|
317
|
-
|
|
318
|
-
#### Native Node `fetch` (with `undici`)
|
|
319
|
-
```javascript
|
|
320
|
-
import { fetch, ProxyAgent } from 'undici';
|
|
321
|
-
import fs from 'fs';
|
|
322
|
-
|
|
323
|
-
const client = new ProxyAgent({
|
|
324
|
-
uri: 'http://127.0.0.1:8888',
|
|
325
|
-
requestTls: { ca: fs.readFileSync('./ca.pem') }
|
|
326
|
-
});
|
|
327
|
-
|
|
328
|
-
const res = await fetch('https://tls.peet.ws/api/all', { dispatcher: client });
|
|
329
|
-
const json = await res.json();
|
|
330
|
-
console.log('JA4:', json.tls.ja4);
|
|
331
|
-
```
|
|
332
|
-
|
|
333
|
-
---
|
|
334
|
-
|
|
335
|
-
### Go (net/http)
|
|
336
|
-
|
|
337
|
-
```go
|
|
338
|
-
package main
|
|
339
|
-
|
|
340
|
-
import (
|
|
341
|
-
"crypto/x509"
|
|
342
|
-
"fmt"
|
|
343
|
-
"io"
|
|
344
|
-
"net/http"
|
|
345
|
-
"net/url"
|
|
346
|
-
"os"
|
|
347
|
-
)
|
|
348
|
-
|
|
349
|
-
func main() {
|
|
350
|
-
caCert, _ := os.ReadFile("ca.pem")
|
|
351
|
-
caCertPool := x509.NewCertPool()
|
|
352
|
-
caCertPool.AppendCertsFromPEM(caCert)
|
|
353
|
-
|
|
354
|
-
proxyURL, _ := url.Parse("http://127.0.0.1:8888")
|
|
355
|
-
transport := &http.Transport{
|
|
356
|
-
Proxy: http.ProxyURL(proxyURL),
|
|
357
|
-
TLSClientConfig: &tls.Config{
|
|
358
|
-
RootCAs: caCertPool,
|
|
359
|
-
},
|
|
360
|
-
}
|
|
361
|
-
|
|
362
|
-
client := &http.Client{Transport: transport}
|
|
363
|
-
resp, err := client.Get("https://tls.peet.ws/api/all")
|
|
364
|
-
if err != nil {
|
|
365
|
-
panic(err)
|
|
366
|
-
}
|
|
367
|
-
defer resp.Body.Close()
|
|
368
|
-
|
|
369
|
-
body, _ := io.ReadAll(resp.Body)
|
|
370
|
-
fmt.Println(string(body))
|
|
371
|
-
}
|
|
372
|
-
```
|
|
373
|
-
|
|
374
|
-
---
|
|
375
|
-
|
|
376
|
-
### Rust (reqwest)
|
|
377
|
-
|
|
378
|
-
```rust
|
|
379
|
-
use reqwest::{Certificate, Client, Proxy};
|
|
380
|
-
use std::fs;
|
|
381
|
-
|
|
382
|
-
#[tokio::main]
|
|
383
|
-
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
384
|
-
let cert_bytes = fs::read("ca.pem")?;
|
|
385
|
-
let cert = Certificate::from_pem(&cert_bytes)?;
|
|
386
|
-
|
|
387
|
-
let proxy = Proxy::all("http://127.0.0.1:8888")?;
|
|
388
|
-
let client = Client::builder()
|
|
389
|
-
.proxy(proxy)
|
|
390
|
-
.add_root_certificate(cert)
|
|
391
|
-
.build()?;
|
|
392
|
-
|
|
393
|
-
let res = client.get("https://tls.peet.ws/api/all").send().await?;
|
|
394
|
-
println!("Response: {}", res.text().await?);
|
|
395
|
-
Ok(())
|
|
396
|
-
}
|
|
397
|
-
```
|
|
398
|
-
|
|
399
|
-
---
|
|
400
|
-
|
|
401
|
-
### Scrapy Framework
|
|
402
|
-
|
|
403
|
-
In your Scrapy project's `settings.py`:
|
|
404
|
-
|
|
405
|
-
```python
|
|
406
|
-
# Enable standard HTTP proxy middleware
|
|
407
|
-
DOWNLOADER_MIDDLEWARES = {
|
|
408
|
-
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 750,
|
|
409
|
-
}
|
|
410
|
-
|
|
411
|
-
# Route all traffic through ja4proxy with Safari 18 profile
|
|
412
|
-
HTTP_PROXY = "http://user-profile_safari18:nopass@127.0.0.1:8888"
|
|
413
|
-
REQUESTS_CA_BUNDLE = "/path/to/ca.pem"
|
|
414
|
-
```
|
|
415
|
-
|
|
416
|
-
---
|
|
417
|
-
|
|
418
|
-
## Dynamic Per-Request Directives
|
|
419
|
-
|
|
420
|
-
`ja4proxy` allows you to customize the browser profile, session pool, or upstream routing on a granular per-request basis without restarting the server.
|
|
421
|
-
|
|
422
|
-
### Via Proxy Credentials (Tag Routing)
|
|
423
|
-
|
|
424
|
-
Pass directives directly inside the proxy username string formatted as `tag_value`:
|
|
425
|
-
|
|
426
|
-
```text
|
|
427
|
-
Format:
|
|
428
|
-
http://[user]-profile_<profile>-session_<id>-proxy_<upstream_url>:[password]@127.0.0.1:8888
|
|
429
|
-
```
|
|
430
|
-
|
|
431
|
-
Examples:
|
|
432
|
-
|
|
433
|
-
1. **Switch to Safari 18**:
|
|
434
|
-
```bash
|
|
435
|
-
curl -x "http://user-profile_safari18:pass@127.0.0.1:8888" --cacert ./ca.pem https://tls.peet.ws/api/all
|
|
436
|
-
```
|
|
437
|
-
|
|
438
|
-
2. **Maintain a Sticky Session Pool**:
|
|
439
|
-
```bash
|
|
440
|
-
curl -x "http://user-profile_chrome133-session_worker12:pass@127.0.0.1:8888" --cacert ./ca.pem https://target.com
|
|
441
|
-
```
|
|
442
|
-
|
|
443
|
-
3. **Chain Out to an Upstream SOCKS5 Proxy**:
|
|
444
|
-
```bash
|
|
445
|
-
curl -x "http://user-profile_chrome133-proxy_socks5://resuser:respass@residential.net:1080:pass@127.0.0.1:8888" --cacert ./ca.pem https://target.com
|
|
446
|
-
```
|
|
447
|
-
|
|
448
|
-
### Via Custom Request Headers
|
|
449
|
-
|
|
450
|
-
When connecting via plain HTTP gateway or custom client hooks, pass directives as standard headers:
|
|
451
|
-
|
|
452
|
-
| Header | Type | Description | Example |
|
|
453
|
-
| :--- | :--- | :--- | :--- |
|
|
454
|
-
| `X-JA4-Profile` | String | Target browser preset | `chrome_133`, `
|
|
455
|
-
| `X-JA4-Session` | String | Persistent session identifier | `client_worker_01` |
|
|
456
|
-
| `X-JA4-Upstream-Proxy` | URL | Forwarding proxy endpoint | `http://user:pass@10.0.0.1:8080` |
|
|
457
|
-
| `X-JA4-Timeout` | Integer | Connection timeout (seconds) | `30` |
|
|
458
|
-
|
|
459
|
-
---
|
|
460
|
-
|
|
461
|
-
## Residential Proxy Chaining
|
|
462
|
-
|
|
463
|
-
`ja4proxy` acts as a TLS and protocol transformer. You can position `ja4proxy` locally on your server or laptop, while routing the actual egress IP through your existing residential or datacenter proxy network:
|
|
464
|
-
|
|
465
|
-
```text
|
|
466
|
-
Client -> ja4proxy (Applies Chrome 133 JA4 + HTTP/2) -> Upstream Residential IP -> Cloudflare
|
|
467
|
-
```
|
|
468
|
-
|
|
469
|
-
To configure global upstream forwarding at startup:
|
|
470
|
-
|
|
471
|
-
```bash
|
|
472
|
-
ja4proxy start --port 8888 --upstream-proxy "http://username:password@residential-provider.com:8000"
|
|
473
|
-
```
|
|
474
|
-
|
|
475
|
-
Or dynamically per request:
|
|
476
|
-
|
|
477
|
-
```python
|
|
478
|
-
import requests
|
|
479
|
-
|
|
480
|
-
proxies = {
|
|
481
|
-
# Dynamically inject residential upstream via tag syntax
|
|
482
|
-
"https": "http://user-proxy_http://resuser:respass@gate.io:8000:pass@127.0.0.1:8888"
|
|
483
|
-
}
|
|
484
|
-
|
|
485
|
-
res = requests.get("https://api.ipify.org?format=json", proxies=proxies, verify="ca.pem")
|
|
486
|
-
print("Egress IP:", res.json()["ip"])
|
|
487
|
-
```
|
|
488
|
-
|
|
489
|
-
---
|
|
490
|
-
|
|
491
|
-
## Supported Browser Presets
|
|
492
|
-
|
|
493
|
-
Run `ja4proxy profiles` to list all supported profiles:
|
|
494
|
-
|
|
495
|
-
| Profile Identifier | Engine | Platform | Key Protocol Features |
|
|
496
|
-
| :--- | :--- | :--- | :--- |
|
|
497
|
-
| `chrome_133` (Default) | Chromium 133 | Windows / macOS | GREASE, ALPN h2, PSK, Post-Quantum Kyber key exchange |
|
|
498
|
-
| `chrome_131` | Chromium 131 | Windows | GREASE, ALPN h2, ChaCha20, AES-GCM |
|
|
499
|
-
| `chrome_124` | Chromium 124 | Windows | GREASE, ALPN h2, TLS 1.3 |
|
|
500
|
-
| `chrome_120` | Chromium 120 | Windows | Standard Chromium TLS 1.3 profile |
|
|
501
|
-
| `safari_16_0` | WebKit | macOS 13 | WebKit TLS 1.3 profile, Safari extension order & H2 framing |
|
|
502
|
-
| `safari_15_6_1` | WebKit | macOS 12 | WebKit TLS 1.3 legacy profile |
|
|
503
|
-
| `safari_ios_16_0` | WebKit iOS | iOS 16 | Mobile WebKit TLS profile |
|
|
504
|
-
| `firefox_133` | Gecko | Windows / Linux | Mozilla extension permutation, no GREASE, specific ALPN order |
|
|
505
|
-
| `firefox_132` | Gecko | Windows | Mozilla TLS 1.3 profile |
|
|
506
|
-
| `firefox_120` | Gecko | Linux | Standard Firefox TLS 1.3 profile |
|
|
507
|
-
| `opera_90` | Chromium / Opera | Windows | Opera Desktop fingerprint |
|
|
508
|
-
| `opera_89` | Chromium / Opera | Windows | Legacy Opera Desktop profile |
|
|
509
|
-
|
|
510
|
-
---
|
|
511
|
-
|
|
512
|
-
## Production Deployment
|
|
513
|
-
|
|
514
|
-
### Docker & Docker Compose
|
|
515
|
-
|
|
516
|
-
#### `docker-compose.yml`
|
|
517
|
-
```yaml
|
|
518
|
-
version: '3.8'
|
|
519
|
-
|
|
520
|
-
services:
|
|
521
|
-
ja4proxy:
|
|
522
|
-
image: ja4proxy:latest
|
|
523
|
-
build:
|
|
524
|
-
context: .
|
|
525
|
-
dockerfile: Dockerfile
|
|
526
|
-
container_name: ja4proxy
|
|
527
|
-
restart: unless-stopped
|
|
528
|
-
ports:
|
|
529
|
-
- "8888:8888"
|
|
530
|
-
volumes:
|
|
531
|
-
- ja4proxy_certs:/root/.ja4proxy
|
|
532
|
-
environment:
|
|
533
|
-
- JA4PROXY_PORT=8888
|
|
534
|
-
- JA4PROXY_PROFILE=chrome_133
|
|
535
|
-
healthcheck:
|
|
536
|
-
test: ["CMD", "curl", "-f", "http://127.0.0.1:8888/https://httpbin.org/status/200"]
|
|
537
|
-
interval: 30s
|
|
538
|
-
timeout: 5s
|
|
539
|
-
retries: 3
|
|
540
|
-
|
|
541
|
-
volumes:
|
|
542
|
-
ja4proxy_certs:
|
|
543
|
-
```
|
|
544
|
-
|
|
545
|
-
Launch with:
|
|
546
|
-
```bash
|
|
547
|
-
docker compose up -d
|
|
548
|
-
```
|
|
549
|
-
|
|
550
|
-
### Systemd Service Unit
|
|
551
|
-
|
|
552
|
-
Create `/etc/systemd/system/ja4proxy.service`:
|
|
553
|
-
|
|
554
|
-
```ini
|
|
555
|
-
[Unit]
|
|
556
|
-
Description=ja4proxy TLS & HTTP/2 Impersonation Proxy
|
|
557
|
-
After=network.target
|
|
558
|
-
|
|
559
|
-
[Service]
|
|
560
|
-
Type=simple
|
|
561
|
-
User=proxyuser
|
|
562
|
-
WorkingDirectory=/opt/ja4proxy
|
|
563
|
-
ExecStart=/usr/local/bin/ja4proxy start --host 0.0.0.0 --port 8888 --profile chrome_133
|
|
564
|
-
Restart=always
|
|
565
|
-
RestartSec=3
|
|
566
|
-
LimitNOFILE=65535
|
|
567
|
-
|
|
568
|
-
[Install]
|
|
569
|
-
WantedBy=multi-user.target
|
|
570
|
-
```
|
|
571
|
-
|
|
572
|
-
Enable and start:
|
|
573
|
-
```bash
|
|
574
|
-
sudo systemctl daemon-reload
|
|
575
|
-
sudo systemctl enable --now ja4proxy
|
|
576
|
-
```
|
|
577
|
-
|
|
578
|
-
### System-Wide CA Trust Installation
|
|
579
|
-
|
|
580
|
-
To use `ja4proxy` transparently without passing `--cacert` or `verify=...` in your scripts:
|
|
581
|
-
|
|
582
|
-
#### Debian / Ubuntu
|
|
583
|
-
```bash
|
|
584
|
-
ja4proxy ca --export /tmp/ja4proxy-ca.crt
|
|
585
|
-
sudo cp /tmp/ja4proxy-ca.crt /usr/local/share/ca-certificates/
|
|
586
|
-
sudo update-ca-certificates
|
|
587
|
-
```
|
|
588
|
-
|
|
589
|
-
#### macOS
|
|
590
|
-
```bash
|
|
591
|
-
ja4proxy ca --export /tmp/ja4proxy-ca.crt
|
|
592
|
-
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain /tmp/ja4proxy-ca.crt
|
|
593
|
-
```
|
|
594
|
-
|
|
595
|
-
#### Windows (PowerShell Administrator)
|
|
596
|
-
```powershell
|
|
597
|
-
ja4proxy ca --export "$env:TEMP\ja4proxy-ca.crt"
|
|
598
|
-
certutil -addstore -f "Root" "$env:TEMP\ja4proxy-ca.crt"
|
|
599
|
-
```
|
|
600
|
-
|
|
601
|
-
---
|
|
602
|
-
|
|
603
|
-
## Security & Engineering Boundaries
|
|
604
|
-
|
|
605
|
-
1. **Pre-Flight SSRF Protection**:
|
|
606
|
-
`ja4proxy` resolves hostnames prior to outbound dispatch and drops requests resolving to private address spaces (RFC 1918 `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`), loopback (`127.0.0.0/8`, `::1`), and cloud metadata (`169.254.169.254`). Internal forwarding can be enabled with `--allow-private-networks`.
|
|
607
|
-
|
|
608
|
-
2. **Decompression & Content-Length Sync**:
|
|
609
|
-
The native Go engine decodes gzip/brotli/deflate responses transparently. `ja4proxy` strips stale `Content-Encoding` and `Transfer-Encoding` headers and recalculates accurate `Content-Length` before serving downstream, preventing double-decompression exceptions.
|
|
610
|
-
|
|
611
|
-
3. **Header Sanitization**:
|
|
612
|
-
Incoming client hop-by-hop headers (`Proxy-Connection`, `Keep-Alive`, `Te`) and automated OpenSSL/Python identifiers are stripped. Canonical browser headers (`User-Agent`, `Sec-CH-UA`, `Sec-CH-UA-Mobile`, `Sec-CH-UA-Platform`) matching the selected profile are automatically synthesized when missing.
|
|
613
|
-
|
|
614
|
-
4. **Zero-GIL Execution**:
|
|
615
|
-
Native C FFI functions execute outside Python's GIL. Concurrency scales across available CPU cores and network threads without interpreter bottlenecks.
|
|
616
|
-
|
|
617
|
-
---
|
|
618
|
-
|
|
619
|
-
## CLI Reference
|
|
620
|
-
|
|
621
|
-
```text
|
|
622
|
-
Usage: ja4proxy [OPTIONS] COMMAND [ARGS]...
|
|
623
|
-
|
|
624
|
-
ja4proxy: High-performance universal forward proxy server with browser TLS (JA4)
|
|
625
|
-
and HTTP/2 impersonation.
|
|
626
|
-
|
|
627
|
-
Options:
|
|
628
|
-
--version Show the version and exit.
|
|
629
|
-
--help Show this message and exit.
|
|
630
|
-
|
|
631
|
-
Commands:
|
|
632
|
-
ca Inspects or exports the local Root CA certificate.
|
|
633
|
-
profiles Lists all supported browser profiles for TLS and HTTP/2.
|
|
634
|
-
start Starts the ja4proxy server.
|
|
635
|
-
test Performs a direct live handshake against tls.peet.ws.
|
|
636
|
-
```
|
|
637
|
-
|
|
638
|
-
### `ja4proxy start` Flags
|
|
639
|
-
|
|
640
|
-
| Option | Default | Description |
|
|
641
|
-
| :--- | :--- | :--- |
|
|
642
|
-
| `--host` | `127.0.0.1` | Network interface to bind |
|
|
643
|
-
| `--port` | `8888` | Port to listen for HTTP, HTTPS CONNECT, and SOCKS5 |
|
|
644
|
-
| `--profile` | `chrome_133` | Default browser profile |
|
|
645
|
-
| `--upstream-proxy` | None | Global upstream forwarding proxy |
|
|
646
|
-
| `--
|
|
647
|
-
| `--
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
|
|
655
|
-
|
|
656
|
-
|
|
657
|
-
|
|
658
|
-
|
|
659
|
-
|
|
660
|
-
|
|
661
|
-
|
|
662
|
-
|
|
663
|
-
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: ja4proxy
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: High-performance universal proxy server with browser TLS (JA4) and HTTP/2 impersonation
|
|
5
|
+
Author-email: cipherwire <dev@cipherwire.link>
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Requires-Python: >=3.10
|
|
8
|
+
Description-Content-Type: text/markdown
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Requires-Dist: cryptography>=41.0.0
|
|
11
|
+
Requires-Dist: click>=8.0.0
|
|
12
|
+
Dynamic: license-file
|
|
13
|
+
|
|
14
|
+
<div align="center">
|
|
15
|
+
|
|
16
|
+
```text
|
|
17
|
+
_ __ __
|
|
18
|
+
(_)___ _/ // / ____ _________ _ ____ __
|
|
19
|
+
/ / __ `/ // /_/ __ \/ ___/ __ \| |/_/ / / /
|
|
20
|
+
/ / /_/ /__ __/ /_/ / / / /_/ /> </ /_/ /
|
|
21
|
+
__/ /\__,_/ /_/ / .___/_/ \____/_/|_|\__, /
|
|
22
|
+
/___/ /_/ /____/
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
### High-performance universal forward proxy with authentic browser TLS (JA4/JA3) and HTTP/2 framing
|
|
26
|
+
|
|
27
|
+
<p>
|
|
28
|
+
<img src="https://img.shields.io/badge/platform-Windows%20%7C%20Linux%20%7C%20macOS-2563eb" alt="Platforms">
|
|
29
|
+
<img src="https://img.shields.io/badge/engine-uTLS%20%2B%20fhttp-4285F4" alt="Engine">
|
|
30
|
+
<img src="https://img.shields.io/badge/protocols-HTTP%20%7C%20HTTPS%20%7C%20SOCKS5-00599C" alt="Protocols">
|
|
31
|
+
<img src="https://img.shields.io/badge/fingerprint-JA4%20%2F%20JA3%20%2F%20Akamai%20H2-6366f1" alt="Fingerprint">
|
|
32
|
+
<img src="https://img.shields.io/badge/license-MIT-green" alt="License: MIT">
|
|
33
|
+
<img src="https://img.shields.io/badge/status-production-22c55e" alt="Status">
|
|
34
|
+
</p>
|
|
35
|
+
|
|
36
|
+
</div>
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Table of Contents
|
|
41
|
+
|
|
42
|
+
- [The Fingerprinting Problem](#the-fingerprinting-problem)
|
|
43
|
+
- [How ja4proxy Works](#how-ja4proxy-works)
|
|
44
|
+
- [Architecture & Protocol Pipeline](#architecture--protocol-pipeline)
|
|
45
|
+
- [Comparative Analysis](#comparative-analysis)
|
|
46
|
+
- [Key Specifications](#key-specifications)
|
|
47
|
+
- [Installation](#installation)
|
|
48
|
+
- [Quick Start](#quick-start)
|
|
49
|
+
- [Multi-Language Client Integration](#multi-language-client-integration)
|
|
50
|
+
- [Python (requests, httpx, aiohttp)](#python-requests-httpx-aiohttp)
|
|
51
|
+
- [Node.js (axios, undici, fetch)](#nodejs-axios-undici-fetch)
|
|
52
|
+
- [Go (net/http)](#go-nethttp)
|
|
53
|
+
- [Rust (reqwest)](#rust-reqwest)
|
|
54
|
+
- [Scrapy Framework](#scrapy-framework)
|
|
55
|
+
- [Dynamic Per-Request Directives](#dynamic-per-request-directives)
|
|
56
|
+
- [Via Proxy Credentials (Tag Routing)](#via-proxy-credentials-tag-routing)
|
|
57
|
+
- [Via Custom Request Headers](#via-custom-request-headers)
|
|
58
|
+
- [Residential Proxy Chaining](#residential-proxy-chaining)
|
|
59
|
+
- [Supported Browser Presets](#supported-browser-presets)
|
|
60
|
+
- [Production Deployment](#production-deployment)
|
|
61
|
+
- [Docker & Docker Compose](#docker--docker-compose)
|
|
62
|
+
- [Systemd Service Unit](#systemd-service-unit)
|
|
63
|
+
- [System-Wide CA Trust Installation](#system-wide-ca-trust-installation)
|
|
64
|
+
- [Security & Engineering Boundaries](#security--engineering-boundaries)
|
|
65
|
+
- [CLI Reference](#cli-reference)
|
|
66
|
+
- [Automated Verification Suite](#automated-verification-suite)
|
|
67
|
+
- [License](#license)
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## The Fingerprinting Problem
|
|
72
|
+
|
|
73
|
+
Modern anti-bot mitigation platforms (Cloudflare Turnstile / Bot Management, Akamai Edge Workers, Imperva, Datadome, Kasada) no longer rely solely on IP reputation. Instead, automated scrapers and bots are blocked at Layer 7 by inspecting client-side protocol artifacts during the connection handshake:
|
|
74
|
+
|
|
75
|
+
```text
|
|
76
|
+
Layer 7 Defense Hierarchy
|
|
77
|
+
├── 1. TLS ClientHello Fingerprinting (JA3 / JA4)
|
|
78
|
+
│ ├── Cipher suites order & availability
|
|
79
|
+
│ ├── Extension list & GREASE (RFC 8701) injection
|
|
80
|
+
│ ├── Elliptic curve groups & signature algorithms
|
|
81
|
+
│ └── ALPN (Application-Layer Protocol Negotiation)
|
|
82
|
+
├── 2. HTTP/2 Transport Fingerprinting (Akamai HTTP2 Hash)
|
|
83
|
+
│ ├── SETTINGS frame parameters (HEADER_TABLE_SIZE, MAX_CONCURRENT_STREAMS)
|
|
84
|
+
│ ├── WINDOW_UPDATE increments
|
|
85
|
+
│ ├── Pseudo-header serialization order (:method, :authority, :scheme, :path)
|
|
86
|
+
│ └── Priority trees and frame dependency graphs
|
|
87
|
+
└── 3. Layer 7 HTTP Header Consistency
|
|
88
|
+
├── Sec-CH-UA, Sec-CH-UA-Mobile, Sec-CH-UA-Platform
|
|
89
|
+
└── User-Agent string alignment with actual cryptographic capabilities
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Standard networking stacks (OpenSSL in Python `urllib3`, Go `crypto/tls`, Node.js `tls`) expose hard-coded, static handshakes that immediately flag requests as non-browser traffic.
|
|
93
|
+
|
|
94
|
+
### Fingerprint Breakdown: OpenSSL vs. ja4proxy (Chrome 133)
|
|
95
|
+
|
|
96
|
+
| Component | Standard Python / Node / Go | ja4proxy (Chrome 133 Profile) |
|
|
97
|
+
| :--- | :--- | :--- |
|
|
98
|
+
| **JA4 Signature** | `t13d030800_...` (Flagged as Bot) | `t13d1516h2_8daaf6152771_d8a2da3f94cd` (Identical to Chrome 133) |
|
|
99
|
+
| **GREASE Support** | None (RFC 8701 absent) | Fully randomized GREASE cipher and extension values |
|
|
100
|
+
| **HTTP Version** | Often forced to HTTP/1.1 | Native HTTP/2 (RFC 7540) with authentic browser stream frames |
|
|
101
|
+
| **Pseudo-Header Order** | Variable or alphabetical | `:method`, `:authority`, `:scheme`, `:path` |
|
|
102
|
+
| **H2 SETTINGS** | Default OpenSSL/Go parameters | `1:65536, 2:0, 4:6291456, 6:262144` matching Chromium |
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## How ja4proxy Works
|
|
107
|
+
|
|
108
|
+
`ja4proxy` is an in-process, high-concurrency forward proxy. Rather than forwarding raw TCP sockets (which passes through your runtime's detectable OpenSSL handshake), `ja4proxy` terminates client requests locally and re-originates the upstream connection using an in-process uTLS and fhttp engine written in Go.
|
|
109
|
+
|
|
110
|
+
```text
|
|
111
|
+
Any Existing Client Application
|
|
112
|
+
(cURL / requests / Scrapy / Playwright / Node / Go / Rust)
|
|
113
|
+
│
|
|
114
|
+
HTTP CONNECT / Plain HTTP / SOCKS5 binary
|
|
115
|
+
│
|
|
116
|
+
▼
|
|
117
|
+
┌────────────────────────────────────────────────────────────────────────┐
|
|
118
|
+
│ ja4proxy Runtime (127.0.0.1:8888) │
|
|
119
|
+
│ │
|
|
120
|
+
│ [Ingress Sniffer] -> Auto-detects protocol by magic byte: │
|
|
121
|
+
│ - 0x05 (SOCKS5 RFC 1928) │
|
|
122
|
+
│ - CONNECT host:443 (HTTPS Tunnel) │
|
|
123
|
+
│ - GET / POST / http://... (HTTP Gateway) │
|
|
124
|
+
│ │
|
|
125
|
+
│ [Directive Parser] -> Extracts profile, session, upstream proxy: │
|
|
126
|
+
│ - Via credential tags: user-profile_chrome133 │
|
|
127
|
+
│ - Via custom headers: X-JA4-Profile │
|
|
128
|
+
│ │
|
|
129
|
+
│ [SSRF Guard] -> Validates target IP against private/loopback CIDRs │
|
|
130
|
+
│ │
|
|
131
|
+
│ [TLS Interceptor] -> Generates dynamic in-memory SAN leaf certs │
|
|
132
|
+
│ │
|
|
133
|
+
│ [Native Zero-GIL FFI] -> Dispatches request into Go uTLS engine │
|
|
134
|
+
└───────────────────────────────────┬────────────────────────────────────┘
|
|
135
|
+
│
|
|
136
|
+
Outbound: Real Chrome 133 JA4 + HTTP/2 Multiplexing
|
|
137
|
+
│
|
|
138
|
+
▼
|
|
139
|
+
Upstream Target (WAF / Protected Host)
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
## Architecture & Protocol Pipeline
|
|
145
|
+
|
|
146
|
+
### 1. Single-Port Protocol Sniffing
|
|
147
|
+
`ja4proxy` binds to a single port and inspects the initial byte stream using non-destructive socket inspection:
|
|
148
|
+
* Byte `0x05`: Dispatches to the binary SOCKS5 state machine (RFC 1928), negotiating `NO AUTH` or `USERNAME/PASSWORD` before taking target host and port.
|
|
149
|
+
* ASCII `CONNECT`: Intercepts HTTPS tunnels, performs an SSL handshake using dynamic per-hostname leaf certificates signed on-the-fly by the local CA, and decrypts request bytes.
|
|
150
|
+
* ASCII `GET / POST / HEAD`: Handles standard forward proxy requests or gateway URLs (`http://127.0.0.1:8888/https://example.com`).
|
|
151
|
+
|
|
152
|
+
### 2. Zero-GIL Native C-Shared FFI
|
|
153
|
+
Python manages client I/O asynchronously via `asyncio`. When dispatching outbound HTTP/TLS requests, `ja4proxy` calls into `libutls` through `ctypes`. The Python Global Interpreter Lock (GIL) is released during socket creation, cryptographic handshakes, and body streaming.
|
|
154
|
+
|
|
155
|
+
### 3. Memory Hygiene & Free Routine
|
|
156
|
+
Native C allocations returned by the Go runtime are wrapped in a structured memory protocol. Once response headers and body buffers are consumed in Python, `freeMemory` is explicitly invoked over the C pointer, preventing memory leaks across long-running proxy processes.
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## Comparative Analysis
|
|
161
|
+
|
|
162
|
+
| Feature | ja4proxy | curl-impersonate | tls-client (Python) | Puppeteer / Playwright | Squid / Privoxy |
|
|
163
|
+
| :--- | :--- | :--- | :--- | :--- | :--- |
|
|
164
|
+
| **Drop-in for Any Language** | Yes (via standard proxy env) | CLI / C only | Python only | Requires browser process | Yes |
|
|
165
|
+
| **Zero Code Changes** | Yes (`HTTP_PROXY=...`) | No (must replace curl binary) | No (rewrite to custom client) | No (heavy refactor) | Yes |
|
|
166
|
+
| **JA4 / JA3 Fingerprint** | Authentic Browser Preset | Authentic Browser Preset | Authentic Browser Preset | Authentic Browser | Leaks OS OpenSSL |
|
|
167
|
+
| **HTTP/2 SETTINGS Impersonation** | Yes | Yes | Yes | Yes | No |
|
|
168
|
+
| **RAM per Worker** | ~18 MB | ~15 MB | ~20 MB | ~150 - 300 MB | ~30 MB |
|
|
169
|
+
| **SOCKS5 Ingress** | Yes (Auto-detected) | No | No | No | Separate port |
|
|
170
|
+
| **Upstream Proxy Chaining** | Dynamic per-request | Static flags | Per-client | Per-browser context | Configuration file |
|
|
171
|
+
|
|
172
|
+
---
|
|
173
|
+
|
|
174
|
+
## Key Specifications
|
|
175
|
+
|
|
176
|
+
- **Listen Protocols**: HTTP Gateway, HTTP CONNECT, SOCKS5 (RFC 1928) on a single port.
|
|
177
|
+
- **TLS Engine**: uTLS with randomized GREASE (RFC 8701) and custom extension order permutations.
|
|
178
|
+
- **HTTP Engine**: fhttp with browser-matched HTTP/2 SETTINGS framing and pseudo-header ordering.
|
|
179
|
+
- **Certificate Authority**: 2048-bit RSA Root CA (`ca.pem`) with automatic in-memory leaf certificate generation and Subject Alternative Name (SAN) caching.
|
|
180
|
+
- **Security Boundary**: Pre-flight connection-time SSRF blocking for RFC 1918 private subnets, loopback, and cloud metadata (`169.254.169.254`).
|
|
181
|
+
- **Decompression Sync**: Automatic header reconciliation (`Content-Encoding`, `Content-Length`) to prevent client decoder desync.
|
|
182
|
+
|
|
183
|
+
---
|
|
184
|
+
|
|
185
|
+
## Installation
|
|
186
|
+
|
|
187
|
+
### Via Source
|
|
188
|
+
|
|
189
|
+
```bash
|
|
190
|
+
git clone https://github.com/cipherwire/ja4proxy.git
|
|
191
|
+
cd ja4proxy
|
|
192
|
+
pip install .
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
### Via Docker
|
|
196
|
+
|
|
197
|
+
```bash
|
|
198
|
+
docker build -t ja4proxy .
|
|
199
|
+
docker run -d -p 8888:8888 --name ja4proxy ja4proxy
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## Quick Start
|
|
205
|
+
|
|
206
|
+
### 1. Launch the Server
|
|
207
|
+
|
|
208
|
+
```bash
|
|
209
|
+
ja4proxy start --host 127.0.0.1 --port 8888 --profile chrome_133
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Output:
|
|
213
|
+
```text
|
|
214
|
+
[*] ja4proxy v0.1.0 starting on http://127.0.0.1:8888
|
|
215
|
+
[*] Default Profile: chrome_133
|
|
216
|
+
[*] Protocols Supported: HTTP CONNECT, HTTP Gateway, SOCKS5 (Auto-detect)
|
|
217
|
+
[*] CA Certificate: C:\Users\AnonNp\.ja4proxy\ca.pem
|
|
218
|
+
[*] Ready for HTTP and SOCKS5 clients
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
### 2. Export the Root CA
|
|
222
|
+
|
|
223
|
+
Export the root certificate so your HTTP client can verify intercept certificates without disabling SSL verification:
|
|
224
|
+
|
|
225
|
+
```bash
|
|
226
|
+
ja4proxy ca --export ./ca.pem
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
### 3. Verify TLS Handshake Against tls.peet.ws
|
|
230
|
+
|
|
231
|
+
#### Using HTTP CONNECT:
|
|
232
|
+
```bash
|
|
233
|
+
curl -x http://127.0.0.1:8888 --cacert ./ca.pem https://tls.peet.ws/api/all
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
#### Using SOCKS5 (Identical Port):
|
|
237
|
+
```bash
|
|
238
|
+
curl -x socks5h://127.0.0.1:8888 --cacert ./ca.pem https://tls.peet.ws/api/all
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
#### Using Gateway Mode (No CA certificate required):
|
|
242
|
+
```bash
|
|
243
|
+
curl http://127.0.0.1:8888/https://tls.peet.ws/api/all
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
Verified output from `tls.peet.ws`:
|
|
247
|
+
```json
|
|
248
|
+
{
|
|
249
|
+
"http_version": "h2",
|
|
250
|
+
"tls": {
|
|
251
|
+
"ja4": "t13d1516h2_8daaf6152771_d8a2da3f94cd",
|
|
252
|
+
"ja3": "771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-21,29-23-24,0"
|
|
253
|
+
}
|
|
254
|
+
}
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
---
|
|
258
|
+
|
|
259
|
+
## Multi-Language Client Integration
|
|
260
|
+
|
|
261
|
+
### Python (requests, httpx, aiohttp)
|
|
262
|
+
|
|
263
|
+
#### `requests`
|
|
264
|
+
```python
|
|
265
|
+
import os
|
|
266
|
+
import requests
|
|
267
|
+
|
|
268
|
+
ca_cert = os.path.expanduser("~/.ja4proxy/ca.pem")
|
|
269
|
+
proxies = {
|
|
270
|
+
"http": "http://127.0.0.1:8888",
|
|
271
|
+
"https": "http://127.0.0.1:8888",
|
|
272
|
+
}
|
|
273
|
+
|
|
274
|
+
# Standard requests call immediately adopts Chrome 133 JA4 fingerprint
|
|
275
|
+
res = requests.get("https://tls.peet.ws/api/all", proxies=proxies, verify=ca_cert)
|
|
276
|
+
print("JA4:", res.json()["tls"]["ja4"])
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
#### `httpx` (Async with SOCKS5)
|
|
280
|
+
```python
|
|
281
|
+
import asyncio
|
|
282
|
+
import os
|
|
283
|
+
import httpx
|
|
284
|
+
|
|
285
|
+
async def main():
|
|
286
|
+
ca_cert = os.path.expanduser("~/.ja4proxy/ca.pem")
|
|
287
|
+
async with httpx.AsyncClient(
|
|
288
|
+
proxy="socks5://127.0.0.1:8888",
|
|
289
|
+
verify=ca_cert,
|
|
290
|
+
) as client:
|
|
291
|
+
res = await client.get("https://tls.peet.ws/api/all")
|
|
292
|
+
print("JA4:", res.json()["tls"]["ja4"])
|
|
293
|
+
|
|
294
|
+
asyncio.run(main())
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
### Node.js (axios, undici, fetch)
|
|
300
|
+
|
|
301
|
+
#### `axios`
|
|
302
|
+
```javascript
|
|
303
|
+
const axios = require('axios');
|
|
304
|
+
const fs = require('fs');
|
|
305
|
+
const https = require('https');
|
|
306
|
+
|
|
307
|
+
const ca = fs.readFileSync('./ca.pem');
|
|
308
|
+
const httpsAgent = new https.Agent({ ca });
|
|
309
|
+
|
|
310
|
+
axios.get('https://tls.peet.ws/api/all', {
|
|
311
|
+
proxy: { host: '127.0.0.1', port: 8888 },
|
|
312
|
+
httpsAgent: httpsAgent
|
|
313
|
+
}).then(res => {
|
|
314
|
+
console.log('JA4:', res.data.tls.ja4);
|
|
315
|
+
});
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
#### Native Node `fetch` (with `undici`)
|
|
319
|
+
```javascript
|
|
320
|
+
import { fetch, ProxyAgent } from 'undici';
|
|
321
|
+
import fs from 'fs';
|
|
322
|
+
|
|
323
|
+
const client = new ProxyAgent({
|
|
324
|
+
uri: 'http://127.0.0.1:8888',
|
|
325
|
+
requestTls: { ca: fs.readFileSync('./ca.pem') }
|
|
326
|
+
});
|
|
327
|
+
|
|
328
|
+
const res = await fetch('https://tls.peet.ws/api/all', { dispatcher: client });
|
|
329
|
+
const json = await res.json();
|
|
330
|
+
console.log('JA4:', json.tls.ja4);
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
---
|
|
334
|
+
|
|
335
|
+
### Go (net/http)
|
|
336
|
+
|
|
337
|
+
```go
|
|
338
|
+
package main
|
|
339
|
+
|
|
340
|
+
import (
|
|
341
|
+
"crypto/x509"
|
|
342
|
+
"fmt"
|
|
343
|
+
"io"
|
|
344
|
+
"net/http"
|
|
345
|
+
"net/url"
|
|
346
|
+
"os"
|
|
347
|
+
)
|
|
348
|
+
|
|
349
|
+
func main() {
|
|
350
|
+
caCert, _ := os.ReadFile("ca.pem")
|
|
351
|
+
caCertPool := x509.NewCertPool()
|
|
352
|
+
caCertPool.AppendCertsFromPEM(caCert)
|
|
353
|
+
|
|
354
|
+
proxyURL, _ := url.Parse("http://127.0.0.1:8888")
|
|
355
|
+
transport := &http.Transport{
|
|
356
|
+
Proxy: http.ProxyURL(proxyURL),
|
|
357
|
+
TLSClientConfig: &tls.Config{
|
|
358
|
+
RootCAs: caCertPool,
|
|
359
|
+
},
|
|
360
|
+
}
|
|
361
|
+
|
|
362
|
+
client := &http.Client{Transport: transport}
|
|
363
|
+
resp, err := client.Get("https://tls.peet.ws/api/all")
|
|
364
|
+
if err != nil {
|
|
365
|
+
panic(err)
|
|
366
|
+
}
|
|
367
|
+
defer resp.Body.Close()
|
|
368
|
+
|
|
369
|
+
body, _ := io.ReadAll(resp.Body)
|
|
370
|
+
fmt.Println(string(body))
|
|
371
|
+
}
|
|
372
|
+
```
|
|
373
|
+
|
|
374
|
+
---
|
|
375
|
+
|
|
376
|
+
### Rust (reqwest)
|
|
377
|
+
|
|
378
|
+
```rust
|
|
379
|
+
use reqwest::{Certificate, Client, Proxy};
|
|
380
|
+
use std::fs;
|
|
381
|
+
|
|
382
|
+
#[tokio::main]
|
|
383
|
+
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
384
|
+
let cert_bytes = fs::read("ca.pem")?;
|
|
385
|
+
let cert = Certificate::from_pem(&cert_bytes)?;
|
|
386
|
+
|
|
387
|
+
let proxy = Proxy::all("http://127.0.0.1:8888")?;
|
|
388
|
+
let client = Client::builder()
|
|
389
|
+
.proxy(proxy)
|
|
390
|
+
.add_root_certificate(cert)
|
|
391
|
+
.build()?;
|
|
392
|
+
|
|
393
|
+
let res = client.get("https://tls.peet.ws/api/all").send().await?;
|
|
394
|
+
println!("Response: {}", res.text().await?);
|
|
395
|
+
Ok(())
|
|
396
|
+
}
|
|
397
|
+
```
|
|
398
|
+
|
|
399
|
+
---
|
|
400
|
+
|
|
401
|
+
### Scrapy Framework
|
|
402
|
+
|
|
403
|
+
In your Scrapy project's `settings.py`:
|
|
404
|
+
|
|
405
|
+
```python
|
|
406
|
+
# Enable standard HTTP proxy middleware
|
|
407
|
+
DOWNLOADER_MIDDLEWARES = {
|
|
408
|
+
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 750,
|
|
409
|
+
}
|
|
410
|
+
|
|
411
|
+
# Route all traffic through ja4proxy with Safari 18 profile
|
|
412
|
+
HTTP_PROXY = "http://user-profile_safari18:nopass@127.0.0.1:8888"
|
|
413
|
+
REQUESTS_CA_BUNDLE = "/path/to/ca.pem"
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
---
|
|
417
|
+
|
|
418
|
+
## Dynamic Per-Request Directives
|
|
419
|
+
|
|
420
|
+
`ja4proxy` allows you to customize the browser profile, session pool, or upstream routing on a granular per-request basis without restarting the server.
|
|
421
|
+
|
|
422
|
+
### Via Proxy Credentials (Tag Routing)
|
|
423
|
+
|
|
424
|
+
Pass directives directly inside the proxy username string formatted as `tag_value`:
|
|
425
|
+
|
|
426
|
+
```text
|
|
427
|
+
Format:
|
|
428
|
+
http://[user]-profile_<profile>-session_<id>-proxy_<upstream_url>:[password]@127.0.0.1:8888
|
|
429
|
+
```
|
|
430
|
+
|
|
431
|
+
Examples:
|
|
432
|
+
|
|
433
|
+
1. **Switch to Safari 18**:
|
|
434
|
+
```bash
|
|
435
|
+
curl -x "http://user-profile_safari18:pass@127.0.0.1:8888" --cacert ./ca.pem https://tls.peet.ws/api/all
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
2. **Maintain a Sticky Session Pool**:
|
|
439
|
+
```bash
|
|
440
|
+
curl -x "http://user-profile_chrome133-session_worker12:pass@127.0.0.1:8888" --cacert ./ca.pem https://target.com
|
|
441
|
+
```
|
|
442
|
+
|
|
443
|
+
3. **Chain Out to an Upstream SOCKS5 Proxy**:
|
|
444
|
+
```bash
|
|
445
|
+
curl -x "http://user-profile_chrome133-proxy_socks5://resuser:respass@residential.net:1080:pass@127.0.0.1:8888" --cacert ./ca.pem https://target.com
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
### Via Custom Request Headers
|
|
449
|
+
|
|
450
|
+
When connecting via plain HTTP gateway or custom client hooks, pass directives as standard headers:
|
|
451
|
+
|
|
452
|
+
| Header | Type | Description | Example |
|
|
453
|
+
| :--- | :--- | :--- | :--- |
|
|
454
|
+
| `X-JA4-Profile` | String | Target browser preset | `chrome_133`, `safari_16_0`, `firefox_133` |
|
|
455
|
+
| `X-JA4-Session` | String | Persistent session identifier | `client_worker_01` |
|
|
456
|
+
| `X-JA4-Upstream-Proxy` | URL | Forwarding proxy endpoint | `http://user:pass@10.0.0.1:8080` |
|
|
457
|
+
| `X-JA4-Timeout` | Integer | Connection timeout (seconds) | `30` |
|
|
458
|
+
|
|
459
|
+
---
|
|
460
|
+
|
|
461
|
+
## Residential Proxy Chaining
|
|
462
|
+
|
|
463
|
+
`ja4proxy` acts as a TLS and protocol transformer. You can position `ja4proxy` locally on your server or laptop, while routing the actual egress IP through your existing residential or datacenter proxy network:
|
|
464
|
+
|
|
465
|
+
```text
|
|
466
|
+
Client -> ja4proxy (Applies Chrome 133 JA4 + HTTP/2) -> Upstream Residential IP -> Cloudflare
|
|
467
|
+
```
|
|
468
|
+
|
|
469
|
+
To configure global upstream forwarding at startup:
|
|
470
|
+
|
|
471
|
+
```bash
|
|
472
|
+
ja4proxy start --port 8888 --upstream-proxy "http://username:password@residential-provider.com:8000"
|
|
473
|
+
```
|
|
474
|
+
|
|
475
|
+
Or dynamically per request:
|
|
476
|
+
|
|
477
|
+
```python
|
|
478
|
+
import requests
|
|
479
|
+
|
|
480
|
+
proxies = {
|
|
481
|
+
# Dynamically inject residential upstream via tag syntax
|
|
482
|
+
"https": "http://user-proxy_http://resuser:respass@gate.io:8000:pass@127.0.0.1:8888"
|
|
483
|
+
}
|
|
484
|
+
|
|
485
|
+
res = requests.get("https://api.ipify.org?format=json", proxies=proxies, verify="ca.pem")
|
|
486
|
+
print("Egress IP:", res.json()["ip"])
|
|
487
|
+
```
|
|
488
|
+
|
|
489
|
+
---
|
|
490
|
+
|
|
491
|
+
## Supported Browser Presets
|
|
492
|
+
|
|
493
|
+
Run `ja4proxy profiles` to list all supported profiles:
|
|
494
|
+
|
|
495
|
+
| Profile Identifier | Engine | Platform | Key Protocol Features |
|
|
496
|
+
| :--- | :--- | :--- | :--- |
|
|
497
|
+
| `chrome_133` (Default) | Chromium 133 | Windows / macOS | GREASE, ALPN h2, PSK, Post-Quantum Kyber key exchange |
|
|
498
|
+
| `chrome_131` | Chromium 131 | Windows | GREASE, ALPN h2, ChaCha20, AES-GCM |
|
|
499
|
+
| `chrome_124` | Chromium 124 | Windows | GREASE, ALPN h2, TLS 1.3 |
|
|
500
|
+
| `chrome_120` | Chromium 120 | Windows | Standard Chromium TLS 1.3 profile |
|
|
501
|
+
| `safari_16_0` | WebKit | macOS 13 | WebKit TLS 1.3 profile, Safari extension order & H2 framing |
|
|
502
|
+
| `safari_15_6_1` | WebKit | macOS 12 | WebKit TLS 1.3 legacy profile |
|
|
503
|
+
| `safari_ios_16_0` | WebKit iOS | iOS 16 | Mobile WebKit TLS profile |
|
|
504
|
+
| `firefox_133` | Gecko | Windows / Linux | Mozilla extension permutation, no GREASE, specific ALPN order |
|
|
505
|
+
| `firefox_132` | Gecko | Windows | Mozilla TLS 1.3 profile |
|
|
506
|
+
| `firefox_120` | Gecko | Linux | Standard Firefox TLS 1.3 profile |
|
|
507
|
+
| `opera_90` | Chromium / Opera | Windows | Opera Desktop fingerprint |
|
|
508
|
+
| `opera_89` | Chromium / Opera | Windows | Legacy Opera Desktop profile |
|
|
509
|
+
|
|
510
|
+
---
|
|
511
|
+
|
|
512
|
+
## Production Deployment
|
|
513
|
+
|
|
514
|
+
### Docker & Docker Compose
|
|
515
|
+
|
|
516
|
+
#### `docker-compose.yml`
|
|
517
|
+
```yaml
|
|
518
|
+
version: '3.8'
|
|
519
|
+
|
|
520
|
+
services:
|
|
521
|
+
ja4proxy:
|
|
522
|
+
image: ja4proxy:latest
|
|
523
|
+
build:
|
|
524
|
+
context: .
|
|
525
|
+
dockerfile: Dockerfile
|
|
526
|
+
container_name: ja4proxy
|
|
527
|
+
restart: unless-stopped
|
|
528
|
+
ports:
|
|
529
|
+
- "8888:8888"
|
|
530
|
+
volumes:
|
|
531
|
+
- ja4proxy_certs:/root/.ja4proxy
|
|
532
|
+
environment:
|
|
533
|
+
- JA4PROXY_PORT=8888
|
|
534
|
+
- JA4PROXY_PROFILE=chrome_133
|
|
535
|
+
healthcheck:
|
|
536
|
+
test: ["CMD", "curl", "-f", "http://127.0.0.1:8888/https://httpbin.org/status/200"]
|
|
537
|
+
interval: 30s
|
|
538
|
+
timeout: 5s
|
|
539
|
+
retries: 3
|
|
540
|
+
|
|
541
|
+
volumes:
|
|
542
|
+
ja4proxy_certs:
|
|
543
|
+
```
|
|
544
|
+
|
|
545
|
+
Launch with:
|
|
546
|
+
```bash
|
|
547
|
+
docker compose up -d
|
|
548
|
+
```
|
|
549
|
+
|
|
550
|
+
### Systemd Service Unit
|
|
551
|
+
|
|
552
|
+
Create `/etc/systemd/system/ja4proxy.service`:
|
|
553
|
+
|
|
554
|
+
```ini
|
|
555
|
+
[Unit]
|
|
556
|
+
Description=ja4proxy TLS & HTTP/2 Impersonation Proxy
|
|
557
|
+
After=network.target
|
|
558
|
+
|
|
559
|
+
[Service]
|
|
560
|
+
Type=simple
|
|
561
|
+
User=proxyuser
|
|
562
|
+
WorkingDirectory=/opt/ja4proxy
|
|
563
|
+
ExecStart=/usr/local/bin/ja4proxy start --host 0.0.0.0 --port 8888 --profile chrome_133
|
|
564
|
+
Restart=always
|
|
565
|
+
RestartSec=3
|
|
566
|
+
LimitNOFILE=65535
|
|
567
|
+
|
|
568
|
+
[Install]
|
|
569
|
+
WantedBy=multi-user.target
|
|
570
|
+
```
|
|
571
|
+
|
|
572
|
+
Enable and start:
|
|
573
|
+
```bash
|
|
574
|
+
sudo systemctl daemon-reload
|
|
575
|
+
sudo systemctl enable --now ja4proxy
|
|
576
|
+
```
|
|
577
|
+
|
|
578
|
+
### System-Wide CA Trust Installation
|
|
579
|
+
|
|
580
|
+
To use `ja4proxy` transparently without passing `--cacert` or `verify=...` in your scripts:
|
|
581
|
+
|
|
582
|
+
#### Debian / Ubuntu
|
|
583
|
+
```bash
|
|
584
|
+
ja4proxy ca --export /tmp/ja4proxy-ca.crt
|
|
585
|
+
sudo cp /tmp/ja4proxy-ca.crt /usr/local/share/ca-certificates/
|
|
586
|
+
sudo update-ca-certificates
|
|
587
|
+
```
|
|
588
|
+
|
|
589
|
+
#### macOS
|
|
590
|
+
```bash
|
|
591
|
+
ja4proxy ca --export /tmp/ja4proxy-ca.crt
|
|
592
|
+
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain /tmp/ja4proxy-ca.crt
|
|
593
|
+
```
|
|
594
|
+
|
|
595
|
+
#### Windows (PowerShell Administrator)
|
|
596
|
+
```powershell
|
|
597
|
+
ja4proxy ca --export "$env:TEMP\ja4proxy-ca.crt"
|
|
598
|
+
certutil -addstore -f "Root" "$env:TEMP\ja4proxy-ca.crt"
|
|
599
|
+
```
|
|
600
|
+
|
|
601
|
+
---
|
|
602
|
+
|
|
603
|
+
## Security & Engineering Boundaries
|
|
604
|
+
|
|
605
|
+
1. **Pre-Flight SSRF Protection**:
|
|
606
|
+
`ja4proxy` resolves hostnames prior to outbound dispatch and drops requests resolving to private address spaces (RFC 1918 `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`), loopback (`127.0.0.0/8`, `::1`), and cloud metadata (`169.254.169.254`). Internal forwarding can be enabled with `--allow-private-networks`.
|
|
607
|
+
|
|
608
|
+
2. **Decompression & Content-Length Sync**:
|
|
609
|
+
The native Go engine decodes gzip/brotli/deflate responses transparently. `ja4proxy` strips stale `Content-Encoding` and `Transfer-Encoding` headers and recalculates accurate `Content-Length` before serving downstream, preventing double-decompression exceptions.
|
|
610
|
+
|
|
611
|
+
3. **Header Sanitization**:
|
|
612
|
+
Incoming client hop-by-hop headers (`Proxy-Connection`, `Keep-Alive`, `Te`) and automated OpenSSL/Python identifiers are stripped. Canonical browser headers (`User-Agent`, `Sec-CH-UA`, `Sec-CH-UA-Mobile`, `Sec-CH-UA-Platform`) matching the selected profile are automatically synthesized when missing.
|
|
613
|
+
|
|
614
|
+
4. **Zero-GIL Execution**:
|
|
615
|
+
Native C FFI functions execute outside Python's GIL. Concurrency scales across available CPU cores and network threads without interpreter bottlenecks.
|
|
616
|
+
|
|
617
|
+
---
|
|
618
|
+
|
|
619
|
+
## CLI Reference
|
|
620
|
+
|
|
621
|
+
```text
|
|
622
|
+
Usage: ja4proxy [OPTIONS] COMMAND [ARGS]...
|
|
623
|
+
|
|
624
|
+
ja4proxy: High-performance universal forward proxy server with browser TLS (JA4)
|
|
625
|
+
and HTTP/2 impersonation.
|
|
626
|
+
|
|
627
|
+
Options:
|
|
628
|
+
--version Show the version and exit.
|
|
629
|
+
--help Show this message and exit.
|
|
630
|
+
|
|
631
|
+
Commands:
|
|
632
|
+
ca Inspects or exports the local Root CA certificate.
|
|
633
|
+
profiles Lists all supported browser profiles for TLS and HTTP/2.
|
|
634
|
+
start Starts the ja4proxy server.
|
|
635
|
+
test Performs a direct live handshake against tls.peet.ws.
|
|
636
|
+
```
|
|
637
|
+
|
|
638
|
+
### `ja4proxy start` Flags
|
|
639
|
+
|
|
640
|
+
| Option | Default | Description |
|
|
641
|
+
| :--- | :--- | :--- |
|
|
642
|
+
| `--host` | `127.0.0.1` | Network interface to bind |
|
|
643
|
+
| `--port` | `8888` | Port to listen for HTTP, HTTPS CONNECT, and SOCKS5 |
|
|
644
|
+
| `--profile` | `chrome_133` | Default browser profile |
|
|
645
|
+
| `--upstream-proxy` | None | Global upstream forwarding proxy |
|
|
646
|
+
| `--auth` | None | Require username:password authentication for clients |
|
|
647
|
+
| `--allow-private-networks` | False | Permits forwarding to private/internal IPs |
|
|
648
|
+
| `--ca-dir` | `~/.ja4proxy` | Custom directory for CA certificates |
|
|
649
|
+
|
|
650
|
+
---
|
|
651
|
+
|
|
652
|
+
## Automated Verification Suite
|
|
653
|
+
|
|
654
|
+
Run the full automated test suite covering CA generation, SOCKS5 state machines, chunked request decoding, SSRF guards, and live upstream verification against `tls.peet.ws` and `httpbin.org`:
|
|
655
|
+
|
|
656
|
+
```bash
|
|
657
|
+
python -m unittest discover -s tests -p "test_*.py" -v
|
|
658
|
+
```
|
|
659
|
+
|
|
660
|
+
All 18 unit and live tests execute with zero warnings and zero regressions:
|
|
661
|
+
```text
|
|
662
|
+
test_ca_creation (test_certs.TestCertificateAuthority) ... ok
|
|
663
|
+
test_leaf_cert_generation (test_certs.TestCertificateAuthority) ... ok
|
|
664
|
+
test_leaf_cert_ip_address (test_certs.TestCertificateAuthority) ... ok
|
|
665
|
+
test_live_handshake_chrome (test_engine.TestNativeEngine) ... ok
|
|
666
|
+
test_live_post_httpbin (test_engine.TestNativeEngine) ... ok
|
|
667
|
+
test_bad_gateway_mapping_status_502 (test_live_proxy.TestLiveProxy) ... ok
|
|
668
|
+
test_chunked_request_upload (test_live_proxy.TestLiveProxy) ... ok
|
|
669
|
+
test_client_proxy_authentication (test_live_proxy.TestLiveProxy) ... ok
|
|
670
|
+
test_gateway_mode_gzip (test_live_proxy.TestLiveProxy) ... ok
|
|
671
|
+
test_safari_ios_headers (test_live_proxy.TestLiveProxy) ... ok
|
|
672
|
+
test_socks5_ingress_tls_peet (test_live_proxy.TestLiveProxy) ... ok
|
|
673
|
+
test_ssrf_protection_blocked (test_live_proxy.TestLiveProxy) ... ok
|
|
674
|
+
test_ssrf_restricted_addresses (test_live_proxy.TestLiveProxy) ... ok
|
|
675
|
+
test_extract_directives_from_headers (test_parser.TestParser) ... ok
|
|
676
|
+
test_extract_directives_priority_header_over_auth (test_parser.TestParser) ... ok
|
|
677
|
+
test_parse_credentials_empty (test_parser.TestParser) ... ok
|
|
678
|
+
test_parse_credentials_proxy_with_hyphens (test_parser.TestParser) ... ok
|
|
679
|
+
test_parse_credentials_tags (test_parser.TestParser) ... ok
|
|
680
|
+
|
|
681
|
+
----------------------------------------------------------------------
|
|
682
|
+
Ran 18 tests in 6.164s
|
|
683
|
+
|
|
684
|
+
OK
|
|
685
|
+
```
|
|
686
|
+
|
|
687
|
+
---
|
|
688
|
+
|
|
689
|
+
## License
|
|
690
|
+
|
|
691
|
+
[MIT](file:///D:/what%20to%20make/LICENSE) License. Designed and engineered by `cipherwire` / `hainahola`.
|