nodetect 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- nodetect-1.0.0/PKG-INFO +364 -0
- nodetect-1.0.0/README.md +338 -0
- nodetect-1.0.0/nodetect/__init__.py +79 -0
- nodetect-1.0.0/nodetect/cli.py +352 -0
- nodetect-1.0.0/nodetect/config.py +138 -0
- nodetect-1.0.0/nodetect/core/__init__.py +16 -0
- nodetect-1.0.0/nodetect/core/clearance.py +66 -0
- nodetect-1.0.0/nodetect/core/cookies.py +58 -0
- nodetect-1.0.0/nodetect/core/engine.py +233 -0
- nodetect-1.0.0/nodetect/core/humanizer.py +149 -0
- nodetect-1.0.0/nodetect/core/iuam.py +58 -0
- nodetect-1.0.0/nodetect/core/patches.py +448 -0
- nodetect-1.0.0/nodetect/core/proxy_pool.py +71 -0
- nodetect-1.0.0/nodetect/core/solver.py +127 -0
- nodetect-1.0.0/nodetect/core/tls_session.py +75 -0
- nodetect-1.0.0/nodetect/core/ua_feed.py +37 -0
- nodetect-1.0.0/nodetect/profile/__init__.py +5 -0
- nodetect-1.0.0/nodetect/profile/fingerprint.py +181 -0
- nodetect-1.0.0/nodetect/profile/harvester.py +66 -0
- nodetect-1.0.0/nodetect/profile/manager.py +184 -0
- nodetect-1.0.0/nodetect.egg-info/PKG-INFO +364 -0
- nodetect-1.0.0/nodetect.egg-info/SOURCES.txt +27 -0
- nodetect-1.0.0/nodetect.egg-info/dependency_links.txt +1 -0
- nodetect-1.0.0/nodetect.egg-info/entry_points.txt +2 -0
- nodetect-1.0.0/nodetect.egg-info/requires.txt +4 -0
- nodetect-1.0.0/nodetect.egg-info/top_level.txt +1 -0
- nodetect-1.0.0/setup.cfg +4 -0
- nodetect-1.0.0/setup.py +35 -0
- nodetect-1.0.0/tests/test_nodetect.py +192 -0
nodetect-1.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,364 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: nodetect
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: High-performance stealth browser automation & persistent profile engine
|
|
5
|
+
Home-page: https://github.com/DEV-MITUL/nodetect
|
|
6
|
+
Author: Mitul Islam
|
|
7
|
+
Author-email: mitulislam483@gmail.com
|
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
|
9
|
+
Classifier: Topic :: Internet :: WWW/HTTP :: Browsers
|
|
10
|
+
Classifier: Topic :: Software Development :: Testing
|
|
11
|
+
Requires-Python: >=3.9
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
Requires-Dist: playwright>=1.40.0
|
|
14
|
+
Requires-Dist: curl_cffi>=0.7.0
|
|
15
|
+
Requires-Dist: click>=8.1.0
|
|
16
|
+
Requires-Dist: rich>=13.0.0
|
|
17
|
+
Dynamic: author
|
|
18
|
+
Dynamic: author-email
|
|
19
|
+
Dynamic: classifier
|
|
20
|
+
Dynamic: description
|
|
21
|
+
Dynamic: description-content-type
|
|
22
|
+
Dynamic: home-page
|
|
23
|
+
Dynamic: requires-dist
|
|
24
|
+
Dynamic: requires-python
|
|
25
|
+
Dynamic: summary
|
|
26
|
+
|
|
27
|
+
<div align="center">
|
|
28
|
+
|
|
29
|
+
# nodetect
|
|
30
|
+
|
|
31
|
+
### Enterprise Anti-Detect Browser Automation & Persistent Profile Orchestration
|
|
32
|
+
|
|
33
|
+
[](https://www.python.org/)
|
|
34
|
+
[](LICENSE)
|
|
35
|
+
[](examples/sannysoft_audit_proof.png)
|
|
36
|
+
[](DOCS.md)
|
|
37
|
+
[](https://t.me/mitul_islam)
|
|
38
|
+
|
|
39
|
+
<br/>
|
|
40
|
+
|
|
41
|
+
```text
|
|
42
|
+
███╗ ██╗ ██████╗ ██████╗ ███████╗████████╗███████╗ ██████╗████████╗
|
|
43
|
+
████╗ ██║██╔═══██╗██╔══██╗██╔════╝╚══██╔══╝██╔════╝██╔════╝╚══██╔══╝
|
|
44
|
+
██╔██╗ ██║██║ ██║██║ ██║█████╗ ██║ █████╗ ██║ ██║
|
|
45
|
+
██║╚██╗██║██║ ██║██║ ██║██╔══╝ ██║ ██╔══╝ ██║ ██║
|
|
46
|
+
██║ ╚████║╚██████╔╝██████╔╝███████╗ ██║ ███████╗╚██████╗ ██║
|
|
47
|
+
╚═╝ ╚═══╝ ╚═════╝ ╚═════╝ ╚══════╝ ╚═╝ ╚══════╝ ╚═════╝ ╚═╝
|
|
48
|
+
E N G I N E - S T E A L T H S U I T E
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
<p align="center">
|
|
52
|
+
<b>Undetected Chromium Runtime</b> •
|
|
53
|
+
<b>High-Entropy Client Hints</b> •
|
|
54
|
+
<b>Bézier Trajectory Dynamics</b> •
|
|
55
|
+
<b>Dual Browser / TLS Pipeline</b> •
|
|
56
|
+
<b>Cloudflare Turnstile Auto-Solver</b>
|
|
57
|
+
</p>
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
<p align="center">
|
|
62
|
+
<a href="https://t.me/mitul_islam">
|
|
63
|
+
<img src="https://cdn.simpleicons.org/telegram/2CA5E0" width="22" height="22" align="absmiddle" alt="Telegram"/>
|
|
64
|
+
<b>Direct Telegram Inquiries:</b> <code>@mitul_islam</code> (https://t.me/mitul_islam)
|
|
65
|
+
</a>
|
|
66
|
+
<br/>
|
|
67
|
+
<sub>Available for enterprise consulting, high-volume scraping infrastructure, and custom WAF evasion (Cloudflare, DataDome, Akamai).</sub>
|
|
68
|
+
</p>
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
</div>
|
|
73
|
+
|
|
74
|
+
## Overview
|
|
75
|
+
|
|
76
|
+
**nodetect** is a production-grade stealth automation framework and persistent profile manager built for Python. Standard automation libraries (raw Playwright, Puppeteer, Selenium) leak dozens of internal automation indicators that cause immediate blocks on Cloudflare, DataDome, Akamai, and Kasada.
|
|
77
|
+
|
|
78
|
+
nodetect eliminates automation markers at the JavaScript prototype level, provides authentic hardware parameter tables, shields your real IP from WebRTC STUN leaks, autonomously solves Cloudflare Turnstile verification checkboxes, and features a hybrid bridge to export authenticated browser sessions into high-speed raw TLS clients (1,000+ requests/sec).
|
|
79
|
+
|
|
80
|
+
For full technical specifications, architecture diagrams, and complete class references, read the **[Comprehensive Documentation (DOCS.md)](DOCS.md)**.
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## Live Audit: Vanilla Playwright vs nodetect Stealth
|
|
85
|
+
|
|
86
|
+
Both engines were evaluated against the industry-standard bot detection suite at `https://bot.sannysoft.com`.
|
|
87
|
+
|
|
88
|
+
| Anti-Bot Detection Vector | Vanilla Playwright | nodetect Stealth | Technical Mitigation |
|
|
89
|
+
| :--- | :---: | :---: | :--- |
|
|
90
|
+
| **`navigator.webdriver`** | Failed (Present) | **Passed (Missing)** | Prototype chain cleaned (`delete Navigator.prototype.webdriver`). No own-property leaks. |
|
|
91
|
+
| **`navigator.userAgentData`** | Failed / Inconsistent | **High-Entropy Hints** | Emulates `getHighEntropyValues()` with matching OS version, architecture, bitness, and brands. |
|
|
92
|
+
| **WebRTC STUN Leak** | Leaks Real Local/ISP IP | **Zero IP Leak Shield** | Intercepts `RTCPeerConnection` SDP offer and strips host candidate IP addresses. |
|
|
93
|
+
| **Cloudflare Turnstile** | Stalls / Blocks | **Autonomous Auto-Solver** | Locates widget in shadow DOM, glides cursor organically along Bézier curve, and solves challenge. |
|
|
94
|
+
| **`window.chrome` Object** | Missing in Headless | **Passed (Present)** | Genuine Chrome runtime with authentic `runtime`, `csi`, and `loadTimes`. |
|
|
95
|
+
| **`Function.prototype.toString`** | Leaks Custom Hooks | **Native Code Protected** | Wraps all patched methods to return `function () { [native code] }`. |
|
|
96
|
+
| **`navigator.plugins` Array** | 0 Items in Headless | **3 Genuine Plugins** | Simulates standard desktop Chrome internal PDF plugins. |
|
|
97
|
+
| **`PluginArray` Prototype** | Fails `instanceof` | **Passed (Valid)** | Constructs genuine `PluginArray.prototype` & `Plugin.prototype` chains with `Symbol.toStringTag`. |
|
|
98
|
+
| **WebGL GPU Identity** | SwiftShader Virtual GPU | **Native Hardware GPU** | Spoofs real unmasked vendors/renderers (AMD Radeon RX, NVIDIA RTX, Apple M2). |
|
|
99
|
+
| **Iframe Nested Contexts** | Leaks Inside Frames | **Isolated & Masked** | Injects stealth propagation directly through `HTMLIFrameElement.prototype.contentWindow`. |
|
|
100
|
+
| **Mouse Trajectory Telemetry** | Linear Instant Snap | **Cubic Bézier Curve** | Organic human movement with ease-in-out deceleration, velocity jitter, and overshoot. |
|
|
101
|
+
| **Typing Latency** | Uniform Delay | **Gaussian Cadence** | Natural human keystroke variance with punctuation pauses. |
|
|
102
|
+
| **Session & Profile State** | Wiped on Process Close | **Encrypted SQLite Store** | Isolated Chromium user data dirs, persistent cookies, and fixed proxy pairing. |
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## Benchmark Proof Artifacts
|
|
107
|
+
|
|
108
|
+
<details open>
|
|
109
|
+
<summary><b>1. Head-to-Head Comparison: Vanilla Playwright vs nodetect (Click to Collapse/Expand)</b></summary>
|
|
110
|
+
<br/>
|
|
111
|
+
|
|
112
|
+
<table>
|
|
113
|
+
<tr>
|
|
114
|
+
<td align="center" width="50%">
|
|
115
|
+
<b>Vanilla Playwright (Flagged as Bot)</b><br/>
|
|
116
|
+
<a href="examples/comparison_vanilla_playwright.png" target="_blank">
|
|
117
|
+
<img src="examples/comparison_vanilla_playwright.png" alt="Vanilla Playwright Bot Detection Failed" width="380" style="max-height: 380px; object-fit: cover; object-position: top; border: 1px solid #ff4444;"/>
|
|
118
|
+
</a>
|
|
119
|
+
<br/>
|
|
120
|
+
<small><i>WebDriver (New) flagged in RED. (Click image to open full resolution)</i></small>
|
|
121
|
+
</td>
|
|
122
|
+
<td align="center" width="50%">
|
|
123
|
+
<b>nodetect Stealth (100% Passed)</b><br/>
|
|
124
|
+
<a href="examples/comparison_nodetect.png" target="_blank">
|
|
125
|
+
<img src="examples/comparison_nodetect.png" alt="nodetect Stealth Audit Passed" width="380" style="max-height: 380px; object-fit: cover; object-position: top; border: 1px solid #00c853;"/>
|
|
126
|
+
</a>
|
|
127
|
+
<br/>
|
|
128
|
+
<small><i>WebDriver (New) missing (passed). (Click image to open full resolution)</i></small>
|
|
129
|
+
</td>
|
|
130
|
+
</tr>
|
|
131
|
+
</table>
|
|
132
|
+
|
|
133
|
+
</details>
|
|
134
|
+
|
|
135
|
+
<details>
|
|
136
|
+
<summary><b>2. Deep Sannysoft Evasion Audit (Full 100/100 Capture)</b></summary>
|
|
137
|
+
<br/>
|
|
138
|
+
|
|
139
|
+
<div align="center">
|
|
140
|
+
<a href="examples/sannysoft_audit_proof.png" target="_blank">
|
|
141
|
+
<img src="examples/sannysoft_audit_proof.png" alt="Full Sannysoft Bot Audit All Green" width="600" style="max-height: 420px; object-fit: cover; object-position: top; border-radius: 6px; border: 1px solid #333;"/>
|
|
142
|
+
</a>
|
|
143
|
+
<br/>
|
|
144
|
+
<small><i>(Click image to view full high-resolution capture)</i></small>
|
|
145
|
+
</div>
|
|
146
|
+
|
|
147
|
+
</details>
|
|
148
|
+
|
|
149
|
+
<details>
|
|
150
|
+
<summary><b>3. Authenticated Login Flow Execution</b></summary>
|
|
151
|
+
<br/>
|
|
152
|
+
|
|
153
|
+
<div align="center">
|
|
154
|
+
<a href="examples/login_success_proof.png" target="_blank">
|
|
155
|
+
<img src="examples/login_success_proof.png" alt="Authenticated Login Flow Screenshot Proof" width="600" style="max-height: 380px; object-fit: cover; border-radius: 6px; border: 1px solid #333;"/>
|
|
156
|
+
</a>
|
|
157
|
+
<br/>
|
|
158
|
+
<small><i>(Click image to view full high-resolution capture)</i></small>
|
|
159
|
+
</div>
|
|
160
|
+
|
|
161
|
+
</details>
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## Quickstart Guide
|
|
166
|
+
|
|
167
|
+
### 1. Installation
|
|
168
|
+
|
|
169
|
+
Install via PyPI:
|
|
170
|
+
```bash
|
|
171
|
+
pip install nodetect
|
|
172
|
+
playwright install chromium
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
Or install from source:
|
|
176
|
+
```bash
|
|
177
|
+
git clone https://github.com/DEV-MITUL/nodetect.git
|
|
178
|
+
cd nodetect
|
|
179
|
+
pip install -e .
|
|
180
|
+
playwright install chromium
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
### 2. Basic Automation Example
|
|
184
|
+
|
|
185
|
+
```python
|
|
186
|
+
import asyncio
|
|
187
|
+
from nodetect import StealthEngine
|
|
188
|
+
|
|
189
|
+
async def main():
|
|
190
|
+
engine = StealthEngine()
|
|
191
|
+
|
|
192
|
+
# Launch undetected browser session (deterministic hardware footprint generated automatically)
|
|
193
|
+
session = await engine.launch(profile_id="account_alpha", headless=False)
|
|
194
|
+
page = session.page
|
|
195
|
+
|
|
196
|
+
# Navigate
|
|
197
|
+
await page.goto("https://duckduckgo.com")
|
|
198
|
+
|
|
199
|
+
# Organic typing with natural Gaussian cadence
|
|
200
|
+
await page.stealth_type("input[name='q']", "nodetect Stealth Browser", wpm=70)
|
|
201
|
+
|
|
202
|
+
# Organic mouse glide along cubic Bézier trajectory
|
|
203
|
+
await page.stealth_click("button[type='submit']")
|
|
204
|
+
|
|
205
|
+
# Persists cookies and session state into SQLite vault
|
|
206
|
+
await session.close()
|
|
207
|
+
|
|
208
|
+
if __name__ == "__main__":
|
|
209
|
+
asyncio.run(main())
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
---
|
|
213
|
+
|
|
214
|
+
## Playwright Interoperability
|
|
215
|
+
|
|
216
|
+
Because `session.page` is a **native Playwright `Page`** and `session.context` is a **native Playwright `BrowserContext`**, you have complete access to the full Playwright API surface without restrictions:
|
|
217
|
+
|
|
218
|
+
```python
|
|
219
|
+
import asyncio
|
|
220
|
+
from nodetect import StealthEngine
|
|
221
|
+
|
|
222
|
+
async def run():
|
|
223
|
+
engine = StealthEngine()
|
|
224
|
+
session = await engine.launch(profile_id="worker_01", headless=False)
|
|
225
|
+
page = session.page
|
|
226
|
+
|
|
227
|
+
# Standard Playwright Locators
|
|
228
|
+
await page.goto("https://target-portal.com")
|
|
229
|
+
await page.locator("#username").fill("admin")
|
|
230
|
+
|
|
231
|
+
# Request Routing & Asset Blocking
|
|
232
|
+
await page.route("**/*.{png,jpg,svg}", lambda route: route.abort())
|
|
233
|
+
|
|
234
|
+
# Multi-tab Contexts
|
|
235
|
+
tab2 = await session.context.new_page()
|
|
236
|
+
await tab2.goto("https://target-portal.com/dashboard")
|
|
237
|
+
|
|
238
|
+
# Native Full-Page Screenshots
|
|
239
|
+
await tab2.screenshot(path="dashboard.png", full_page=True)
|
|
240
|
+
|
|
241
|
+
await session.close()
|
|
242
|
+
|
|
243
|
+
asyncio.run(run())
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
---
|
|
247
|
+
|
|
248
|
+
## The Hybrid Pipeline: Solve Once, Scrape at 1,000 req/s
|
|
249
|
+
|
|
250
|
+
Running dozens of browser instances consumes massive server memory. nodetect allows solving the initial challenge or login in an undetected browser, then syncing cookies directly to a raw TLS client:
|
|
251
|
+
|
|
252
|
+
```python
|
|
253
|
+
import asyncio
|
|
254
|
+
from nodetect import StealthEngine
|
|
255
|
+
|
|
256
|
+
async def hybrid_scrape():
|
|
257
|
+
engine = StealthEngine()
|
|
258
|
+
session = await engine.launch(profile_id="harvester_01", headless=True)
|
|
259
|
+
page = session.page
|
|
260
|
+
|
|
261
|
+
# 1. Solve challenge / authenticate in browser
|
|
262
|
+
await page.goto("https://target-portal.com/login")
|
|
263
|
+
|
|
264
|
+
# 2. Bridge authenticated cookies into raw curl_cffi TLS session
|
|
265
|
+
tls = await session.export_to_tls_session()
|
|
266
|
+
await session.close()
|
|
267
|
+
|
|
268
|
+
# 3. Fire lightning-fast requests matching real Chrome TLS/JA3 fingerprints
|
|
269
|
+
response = tls.get("https://target-portal.com/api/data")
|
|
270
|
+
print("Fast TLS Response Status:", response.status_code)
|
|
271
|
+
tls.close()
|
|
272
|
+
|
|
273
|
+
asyncio.run(hybrid_scrape())
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
---
|
|
277
|
+
|
|
278
|
+
## Cloudflare Turnstile & Clearance Harvesting
|
|
279
|
+
|
|
280
|
+
### Autonomous Turnstile Solver
|
|
281
|
+
```python
|
|
282
|
+
# Automatically locate Turnstile widget, glide cursor, and click
|
|
283
|
+
solved = await page.solve_turnstile(timeout_sec=20)
|
|
284
|
+
|
|
285
|
+
# Extract verified response token
|
|
286
|
+
token = await page.harvest_turnstile_token()
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
### Acquiring `cf_clearance` via Terminal CLI
|
|
290
|
+
```bash
|
|
291
|
+
python -m nodetect.cli clearance https://nowsecure.nl
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
---
|
|
295
|
+
|
|
296
|
+
## Terminal Command-Line Suite
|
|
297
|
+
|
|
298
|
+
nodetect includes a terminal interface for profile management and benchmark auditing:
|
|
299
|
+
|
|
300
|
+
```bash
|
|
301
|
+
# View system architecture status
|
|
302
|
+
python -m nodetect.cli info
|
|
303
|
+
|
|
304
|
+
# Create isolated profile bound to a proxy
|
|
305
|
+
python -m nodetect.cli profile create worker_01 --proxy "http://user:pass@host:port"
|
|
306
|
+
|
|
307
|
+
# Import real consumer hardware dump (harvest-fp.html)
|
|
308
|
+
python -m nodetect.cli profile import laptop_01 ./real-fp.json
|
|
309
|
+
|
|
310
|
+
# List registered profiles & hardware configurations
|
|
311
|
+
python -m nodetect.cli profile list
|
|
312
|
+
|
|
313
|
+
# Launch interactive browser session (auto-saves state on close)
|
|
314
|
+
python -m nodetect.cli launch worker_01 --url "https://twitter.com/login"
|
|
315
|
+
|
|
316
|
+
# Run automated detection audit
|
|
317
|
+
python -m nodetect.cli test --profile worker_01 --headless
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
---
|
|
321
|
+
|
|
322
|
+
## Verification & Test Suite
|
|
323
|
+
|
|
324
|
+
Run the unit test suite to verify fingerprint determinism, SQLite persistence, and evasion compilation:
|
|
325
|
+
|
|
326
|
+
```bash
|
|
327
|
+
pytest tests/
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
```text
|
|
331
|
+
tests/test_nodetect.py ........ [100%]
|
|
332
|
+
============================== 8 passed in 0.92s ==============================
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
---
|
|
336
|
+
|
|
337
|
+
## Documentation
|
|
338
|
+
|
|
339
|
+
For the complete technical manual covering all classes, methods, parameters, and architecture blueprints, refer to:
|
|
340
|
+
|
|
341
|
+
- **[Full Documentation Manual (DOCS.md)](DOCS.md)**
|
|
342
|
+
|
|
343
|
+
---
|
|
344
|
+
|
|
345
|
+
## Commercial Support & Engineering
|
|
346
|
+
|
|
347
|
+
For enterprise infrastructure, custom scraping clusters, or bespoke WAF bypass:
|
|
348
|
+
|
|
349
|
+
<p align="left">
|
|
350
|
+
<a href="https://t.me/mitul_islam">
|
|
351
|
+
<img src="https://img.shields.io/badge/Telegram-Direct%20Chat-2CA5E0?style=for-the-badge&logo=telegram&logoColor=white" alt="Telegram Contact"/>
|
|
352
|
+
</a>
|
|
353
|
+
</p>
|
|
354
|
+
|
|
355
|
+
- **Lead Engineer:** Mitul Islam
|
|
356
|
+
- **Telegram Handle:** [`@mitul_islam`](https://t.me/mitul_islam) (`https://t.me/mitul_islam`)
|
|
357
|
+
- **Email:** `mitulislam483@gmail.com`
|
|
358
|
+
- **GitHub:** [`DEV-MITUL`](https://github.com/DEV-MITUL)
|
|
359
|
+
|
|
360
|
+
<br/>
|
|
361
|
+
|
|
362
|
+
<div align="center">
|
|
363
|
+
<sub>Distributed under the MIT License. Built for precision automation, research, and data infrastructure.</sub>
|
|
364
|
+
</div>
|