grewbrowser 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- grewbrowser-0.1.0/.gitignore +23 -0
- grewbrowser-0.1.0/CHANGELOG.md +49 -0
- grewbrowser-0.1.0/LICENSE +21 -0
- grewbrowser-0.1.0/PKG-INFO +531 -0
- grewbrowser-0.1.0/README.md +469 -0
- grewbrowser-0.1.0/docs/images/banner.svg +17 -0
- grewbrowser-0.1.0/docs/images/flow.svg +68 -0
- grewbrowser-0.1.0/docs/images/pointer-path.svg +62 -0
- grewbrowser-0.1.0/docs/images/selfcheck.svg +58 -0
- grewbrowser-0.1.0/docs/images/wheel.svg +9 -0
- grewbrowser-0.1.0/examples/demo_extension/background.js +3 -0
- grewbrowser-0.1.0/examples/demo_extension/content.js +3 -0
- grewbrowser-0.1.0/examples/demo_extension/manifest.json +16 -0
- grewbrowser-0.1.0/examples/open_browser.py +21 -0
- grewbrowser-0.1.0/examples/quickstart.py +88 -0
- grewbrowser-0.1.0/examples/with_extension.py +48 -0
- grewbrowser-0.1.0/grewbrowser/__init__.py +101 -0
- grewbrowser-0.1.0/grewbrowser/__main__.py +248 -0
- grewbrowser-0.1.0/grewbrowser/_driver.py +61 -0
- grewbrowser-0.1.0/grewbrowser/behavior/__init__.py +43 -0
- grewbrowser-0.1.0/grewbrowser/behavior/config.py +197 -0
- grewbrowser-0.1.0/grewbrowser/behavior/curves.py +266 -0
- grewbrowser-0.1.0/grewbrowser/behavior/human.py +657 -0
- grewbrowser-0.1.0/grewbrowser/behavior/keyboard.py +199 -0
- grewbrowser-0.1.0/grewbrowser/behavior/world.py +264 -0
- grewbrowser-0.1.0/grewbrowser/core/__init__.py +69 -0
- grewbrowser-0.1.0/grewbrowser/core/binary_manager.py +382 -0
- grewbrowser-0.1.0/grewbrowser/core/download.py +548 -0
- grewbrowser-0.1.0/grewbrowser/core/flags.py +253 -0
- grewbrowser-0.1.0/grewbrowser/core/profile.py +407 -0
- grewbrowser-0.1.0/grewbrowser/diagnostics.py +617 -0
- grewbrowser-0.1.0/grewbrowser/exceptions.py +111 -0
- grewbrowser-0.1.0/grewbrowser/fingerprint/__init__.py +21 -0
- grewbrowser-0.1.0/grewbrowser/fingerprint/applier.py +206 -0
- grewbrowser-0.1.0/grewbrowser/fingerprint/scripts.py +483 -0
- grewbrowser-0.1.0/grewbrowser/launcher.py +951 -0
- grewbrowser-0.1.0/grewbrowser/network/__init__.py +20 -0
- grewbrowser-0.1.0/grewbrowser/network/geoip.py +379 -0
- grewbrowser-0.1.0/grewbrowser/network/proxy.py +144 -0
- grewbrowser-0.1.0/grewbrowser/network/relay.py +419 -0
- grewbrowser-0.1.0/grewbrowser/session/__init__.py +17 -0
- grewbrowser-0.1.0/grewbrowser/session/pool.py +343 -0
- grewbrowser-0.1.0/grewbrowser/session/teardown.py +317 -0
- grewbrowser-0.1.0/pyproject.toml +104 -0
- grewbrowser-0.1.0/tests/conftest.py +45 -0
- grewbrowser-0.1.0/tests/test_live.py +234 -0
- grewbrowser-0.1.0/tests/test_offline.py +340 -0
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
__pycache__/
|
|
2
|
+
*.py[cod]
|
|
3
|
+
*.egg-info/
|
|
4
|
+
.eggs/
|
|
5
|
+
build/
|
|
6
|
+
dist/
|
|
7
|
+
.venv/
|
|
8
|
+
venv/
|
|
9
|
+
env/
|
|
10
|
+
.pytest_cache/
|
|
11
|
+
.mypy_cache/
|
|
12
|
+
.ruff_cache/
|
|
13
|
+
.coverage
|
|
14
|
+
htmlcov/
|
|
15
|
+
examples/profiles/
|
|
16
|
+
profiles/
|
|
17
|
+
.grewbrowser/
|
|
18
|
+
.DS_Store
|
|
19
|
+
Thumbs.db
|
|
20
|
+
*.log
|
|
21
|
+
|
|
22
|
+
examples/deof/
|
|
23
|
+
examples/extensions/
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here. The format follows
|
|
4
|
+
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
|
|
5
|
+
[Semantic Versioning](https://semver.org/).
|
|
6
|
+
|
|
7
|
+
## [0.1.0] — 2026-11-14
|
|
8
|
+
|
|
9
|
+
First release. 🎉
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- **Own managed browser** — `grewbrowser install` fetches a full Chromium from
|
|
14
|
+
Google's Chrome for Testing host into `~/.grewbrowser`. No
|
|
15
|
+
`playwright install` required, and a fresh machine downloads it automatically
|
|
16
|
+
on the first launch (`GREWBROWSER_AUTO_DOWNLOAD=0` turns that off).
|
|
17
|
+
- **Four launchers** — `launch`, `launch_async`,
|
|
18
|
+
`launch_persistent_context`, `launch_persistent_context_async`, each
|
|
19
|
+
returning a session that owns its driver, browser, context, relay and
|
|
20
|
+
profile directory.
|
|
21
|
+
- **One seed, one identity** — `BrowserProfile` derives the user agent, client
|
|
22
|
+
hints, platform, screen, viewport, GPU strings, core count and memory from a
|
|
23
|
+
single seed; a persistent profile pins that seed in
|
|
24
|
+
`<user_data_dir>/.grewbrowser-seed`.
|
|
25
|
+
- **Stealth flags** — automation flags suppressed, hardware ANGLE backend per
|
|
26
|
+
platform (D3D11 / Metal / GL), GPU blocklist bypassed, WebRTC restricted to
|
|
27
|
+
the proxy, renderer backgrounding and timer throttling disabled.
|
|
28
|
+
- **Fingerprint injection** — prototype-level patches with native-looking
|
|
29
|
+
`toString`, seed-deterministic canvas and audio noise, a `chrome` object that
|
|
30
|
+
matches a real Chrome (including the absence of `chrome.runtime`), and
|
|
31
|
+
patched iframe realms.
|
|
32
|
+
- **Human input layer** — `human_click`, `human_type`, `human_fill`,
|
|
33
|
+
`human_scroll`, `human_move`, `human_idle`: Bézier pointer paths with
|
|
34
|
+
Fitts'-law timing and corrective sub-strokes, keystroke intervals that depend
|
|
35
|
+
on the key pair with corrected typos, and wheel scrolling in real platform
|
|
36
|
+
notches. Actionability probes run in a CDP isolated world.
|
|
37
|
+
- **Network identity** — proxy exit-IP lookup that sets timezone, locale and
|
|
38
|
+
coordinates to match; a loopback relay that holds proxy credentials so the
|
|
39
|
+
browser never sees a 407.
|
|
40
|
+
- **Session management** — `BrowserPool` and `SyncBrowserPool` with a
|
|
41
|
+
concurrency semaphore and a launch-rate token bucket; a teardown guard with
|
|
42
|
+
`atexit` and signal hooks that kills the whole browser process tree.
|
|
43
|
+
- **Self-check** — `grewbrowser check` runs 110 probes per mode and prints
|
|
44
|
+
headless against headful side by side.
|
|
45
|
+
- **One driver per thread** — several synchronous sessions can be open at once
|
|
46
|
+
in the same thread; they share a reference-counted Playwright driver that
|
|
47
|
+
stops when the last session closes.
|
|
48
|
+
- **Platforms** — Windows, Linux and macOS, with CI running the full suite on
|
|
49
|
+
all three.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 X2Super
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,531 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: grewbrowser
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Stealth Chromium for Playwright: coherent fingerprints, geo-matched identities, human-paced input, and its own managed browser.
|
|
5
|
+
Project-URL: Homepage, https://github.com/x2super/grewbrowser
|
|
6
|
+
Project-URL: Documentation, https://github.com/x2super/grewbrowser#readme
|
|
7
|
+
Project-URL: Source, https://github.com/x2super/grewbrowser
|
|
8
|
+
Project-URL: Issues, https://github.com/x2super/grewbrowser/issues
|
|
9
|
+
Project-URL: Changelog, https://github.com/x2super/grewbrowser/blob/main/CHANGELOG.md
|
|
10
|
+
Author-email: x2super <august260853@gmail.com>
|
|
11
|
+
Maintainer-email: x2super <august260853@gmail.com>
|
|
12
|
+
License: MIT License
|
|
13
|
+
|
|
14
|
+
Copyright (c) 2026 X2Super
|
|
15
|
+
|
|
16
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
17
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
18
|
+
in the Software without restriction, including without limitation the rights
|
|
19
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
20
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
21
|
+
furnished to do so, subject to the following conditions:
|
|
22
|
+
|
|
23
|
+
The above copyright notice and this permission notice shall be included in all
|
|
24
|
+
copies or substantial portions of the Software.
|
|
25
|
+
|
|
26
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
27
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
28
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
29
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
30
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
31
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
32
|
+
SOFTWARE.
|
|
33
|
+
License-File: LICENSE
|
|
34
|
+
Keywords: automation,browser,chromium,fingerprint,playwright,scraping,stealth,testing
|
|
35
|
+
Classifier: Development Status :: 4 - Beta
|
|
36
|
+
Classifier: Intended Audience :: Developers
|
|
37
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
38
|
+
Classifier: Operating System :: MacOS :: MacOS X
|
|
39
|
+
Classifier: Operating System :: Microsoft :: Windows
|
|
40
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
41
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
42
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
43
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
44
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
45
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
46
|
+
Classifier: Topic :: Internet :: WWW/HTTP :: Browsers
|
|
47
|
+
Classifier: Topic :: Software Development :: Testing
|
|
48
|
+
Classifier: Typing :: Typed
|
|
49
|
+
Requires-Python: >=3.10
|
|
50
|
+
Requires-Dist: playwright>=1.45
|
|
51
|
+
Provides-Extra: dev
|
|
52
|
+
Requires-Dist: build>=1.2; extra == 'dev'
|
|
53
|
+
Requires-Dist: mypy>=1.10; extra == 'dev'
|
|
54
|
+
Requires-Dist: psutil>=5.9; extra == 'dev'
|
|
55
|
+
Requires-Dist: pytest-timeout>=2.3; extra == 'dev'
|
|
56
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
57
|
+
Requires-Dist: ruff>=0.5; extra == 'dev'
|
|
58
|
+
Requires-Dist: twine>=5.1; extra == 'dev'
|
|
59
|
+
Provides-Extra: process
|
|
60
|
+
Requires-Dist: psutil>=5.9; extra == 'process'
|
|
61
|
+
Description-Content-Type: text/markdown
|
|
62
|
+
|
|
63
|
+
<p align="center">
|
|
64
|
+
<img src="docs/images/banner.svg" alt="grewbrowser" width="100%">
|
|
65
|
+
</p>
|
|
66
|
+
|
|
67
|
+
<p align="center">
|
|
68
|
+
<a href="https://github.com/x2super/grewbrowser/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/x2super/grewbrowser/actions/workflows/ci.yml/badge.svg"></a>
|
|
69
|
+
<a href="https://pypi.org/project/grewbrowser/"><img alt="PyPI" src="https://img.shields.io/pypi/v/grewbrowser.svg?color=1f6feb"></a>
|
|
70
|
+
<a href="https://pypi.org/project/grewbrowser/"><img alt="Python" src="https://img.shields.io/pypi/pyversions/grewbrowser.svg"></a>
|
|
71
|
+
<img alt="Platforms" src="https://img.shields.io/badge/platform-windows%20%7C%20linux%20%7C%20macos-8957e5">
|
|
72
|
+
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-3fb950"></a>
|
|
73
|
+
</p>
|
|
74
|
+
|
|
75
|
+
<p align="center">
|
|
76
|
+
<b>Stealth Chromium for Playwright.</b><br>
|
|
77
|
+
One call gives you a browser whose flags, fingerprint, timezone, locale and input behaviour<br>
|
|
78
|
+
all describe the same machine in the same place — instead of a default automation profile with patches bolted on.
|
|
79
|
+
</p>
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## ⚡ Sixty seconds
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
pip install grewbrowser
|
|
87
|
+
grewbrowser install
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
```python
|
|
91
|
+
from grewbrowser import launch
|
|
92
|
+
|
|
93
|
+
with launch() as session:
|
|
94
|
+
page = session.new_page()
|
|
95
|
+
page.goto("https://example.com")
|
|
96
|
+
|
|
97
|
+
page.human_click("#login")
|
|
98
|
+
page.human_type("#email", "someone@example.com")
|
|
99
|
+
page.human_scroll(600)
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
That is the whole setup. No `playwright install`, no flag lists, no fingerprint JSON to maintain.
|
|
103
|
+
|
|
104
|
+
> 💡 **Fresh machine?** You do not even need `grewbrowser install` — the first
|
|
105
|
+
> `launch()` fetches the browser automatically. Set `GREWBROWSER_AUTO_DOWNLOAD=0`
|
|
106
|
+
> if you would rather it never touched the network.
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## ✨ What actually changes
|
|
111
|
+
|
|
112
|
+
| | Plain Playwright | grewbrowser |
|
|
113
|
+
|---|---|---|
|
|
114
|
+
| 🤖 **Automation flags** | `--enable-automation`, `navigator.webdriver` set | suppressed, no infobar, no `webdriver` |
|
|
115
|
+
| 🎮 **GPU** | SwiftShader — a renderer string no real user has | hardware ANGLE: D3D11 / Metal / GL, blocklist bypassed |
|
|
116
|
+
| 📡 **WebRTC** | UDP candidates escape the proxy | `disable_non_proxied_udp` + permission check |
|
|
117
|
+
| 🔑 **Proxy auth** | a `407` round trip on every tunnel | credentials held by a loopback relay, zero challenges |
|
|
118
|
+
| 🌍 **Timezone & locale** | your machine's | resolved from the proxy's **exit** address |
|
|
119
|
+
| 🖱️ **`click()`** | teleports to the exact centre, presses instantly | Bézier path, ~60 Hz events, overshoot + correction, aim pause |
|
|
120
|
+
| ⌨️ **`fill()`** | sets `value`, zero key events | per-key intervals that depend on the key pair, typos that get fixed |
|
|
121
|
+
| 🖲️ **`wheel(0, 2000)`** | one impossible 2000px event | real platform notches with ramp-up and ramp-down |
|
|
122
|
+
| 🔍 **Actionability checks** | run in the page's own world | run in a CDP **isolated world** the page cannot see |
|
|
123
|
+
| 💥 **Crash** | leaked browser + profile directory | `atexit` + signal hooks kill the whole process tree |
|
|
124
|
+
| 🧪 **Proof** | — | `grewbrowser check`: 110 probes, headless vs headful |
|
|
125
|
+
|
|
126
|
+
---
|
|
127
|
+
|
|
128
|
+
## 📦 Install
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
pip install grewbrowser
|
|
132
|
+
pip install "grewbrowser[process]"
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
`[process]` adds `psutil`, which lets the teardown guard kill a browser process
|
|
136
|
+
tree even if the driver dies first. Recommended for anything long-running.
|
|
137
|
+
|
|
138
|
+
### Getting a browser 🌐
|
|
139
|
+
|
|
140
|
+
grewbrowser manages its own Chromium, so a fresh machine needs nothing else:
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
grewbrowser install
|
|
144
|
+
grewbrowser install --version 157.0.8103.78
|
|
145
|
+
grewbrowser install --source chromium-snapshots
|
|
146
|
+
grewbrowser list
|
|
147
|
+
grewbrowser info
|
|
148
|
+
grewbrowser clean
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
Builds land in `~/.grewbrowser` (override with `GREWBROWSER_CACHE_DIR`) and come
|
|
152
|
+
from Google's official Chrome for Testing host.
|
|
153
|
+
|
|
154
|
+
### Picking a browser 🎛️
|
|
155
|
+
|
|
156
|
+
| `channel=` | What it uses | Downloads? |
|
|
157
|
+
|---|---|---|
|
|
158
|
+
| `"auto"` *(default)* | cached own build → Playwright Chromium → system Chrome → Edge | only if nothing at all is found |
|
|
159
|
+
| `"own"` | grewbrowser's own Chromium | yes, on demand |
|
|
160
|
+
| `"chromium"` | Playwright's Chromium **only** | no — fails with a hint |
|
|
161
|
+
| `"chrome"` / `"chrome-beta"` | that system install only | no |
|
|
162
|
+
| `"chromium-system"` | `/usr/bin/chromium` and friends | no |
|
|
163
|
+
| `"msedge"` | system Edge | no |
|
|
164
|
+
| `executable_path=...` | your own binary | no |
|
|
165
|
+
|
|
166
|
+
Named channels are strict on purpose: asking for Chromium and silently getting
|
|
167
|
+
Chrome would mean a different build, version and GPU strings than you asked for.
|
|
168
|
+
|
|
169
|
+
---
|
|
170
|
+
|
|
171
|
+
## 🚀 Usage
|
|
172
|
+
|
|
173
|
+
<details open>
|
|
174
|
+
<summary><b>Sync</b></summary>
|
|
175
|
+
|
|
176
|
+
```python
|
|
177
|
+
from grewbrowser import launch
|
|
178
|
+
|
|
179
|
+
with launch(headless=True, proxy="http://user:pass@gateway:8080") as session:
|
|
180
|
+
print(session.profile.timezone_id, session.profile.locale)
|
|
181
|
+
page = session.new_page()
|
|
182
|
+
page.goto("https://example.com")
|
|
183
|
+
page.human_click("button[type=submit]")
|
|
184
|
+
```
|
|
185
|
+
</details>
|
|
186
|
+
|
|
187
|
+
<details>
|
|
188
|
+
<summary><b>Async</b></summary>
|
|
189
|
+
|
|
190
|
+
```python
|
|
191
|
+
import asyncio
|
|
192
|
+
from grewbrowser import launch_async
|
|
193
|
+
|
|
194
|
+
async def main():
|
|
195
|
+
session = await launch_async(headless=True)
|
|
196
|
+
try:
|
|
197
|
+
page = await session.new_page()
|
|
198
|
+
await page.goto("https://example.com")
|
|
199
|
+
await page.human_type("#q", "hello")
|
|
200
|
+
await page.human_scroll(500)
|
|
201
|
+
finally:
|
|
202
|
+
await session.close()
|
|
203
|
+
|
|
204
|
+
asyncio.run(main())
|
|
205
|
+
```
|
|
206
|
+
</details>
|
|
207
|
+
|
|
208
|
+
<details>
|
|
209
|
+
<summary><b>Persistent identity</b></summary>
|
|
210
|
+
|
|
211
|
+
```python
|
|
212
|
+
from grewbrowser import launch_persistent_context
|
|
213
|
+
|
|
214
|
+
session = launch_persistent_context("./profiles/alice", headless=False)
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
Cookies **and** fingerprint live together: the seed is pinned in
|
|
218
|
+
`<user_data_dir>/.grewbrowser-seed` on the first run and read back afterwards.
|
|
219
|
+
Delete that one file to re-roll the identity while keeping the cookies.
|
|
220
|
+
</details>
|
|
221
|
+
|
|
222
|
+
<details>
|
|
223
|
+
<summary><b>Many at once</b></summary>
|
|
224
|
+
|
|
225
|
+
```python
|
|
226
|
+
from grewbrowser import BrowserPool
|
|
227
|
+
|
|
228
|
+
async def scrape(session, url):
|
|
229
|
+
page = await session.new_page()
|
|
230
|
+
await page.goto(url)
|
|
231
|
+
return await page.title()
|
|
232
|
+
|
|
233
|
+
async with BrowserPool(max_concurrent=4, launches_per_minute=30, proxy=PROXY) as pool:
|
|
234
|
+
titles = await pool.map(urls, scrape)
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
`max_concurrent` bounds live browsers; `launches_per_minute` bounds cold starts
|
|
238
|
+
— a separate problem, because a hundred simultaneous starts thrash the disk even
|
|
239
|
+
when a hundred live browsers would fit in memory. `SyncBrowserPool` does the
|
|
240
|
+
same across a thread pool.
|
|
241
|
+
</details>
|
|
242
|
+
|
|
243
|
+
---
|
|
244
|
+
|
|
245
|
+
## 🧬 How one launch works
|
|
246
|
+
|
|
247
|
+
<p align="center"><img src="docs/images/flow.svg" alt="launch pipeline" width="100%"></p>
|
|
248
|
+
|
|
249
|
+
The order is the point. Each step feeds the next, so nothing has to be patched
|
|
250
|
+
after the fact:
|
|
251
|
+
|
|
252
|
+
1. 🔌 **Proxy** — parsed once into a form that can produce a *credential-free* URL.
|
|
253
|
+
2. 🌍 **GeoIP** — the exit address is resolved **before** the identity is built, so
|
|
254
|
+
timezone and locale are part of it rather than bolted on after the first page
|
|
255
|
+
already loaded. `geoip="strict"` raises instead of warning.
|
|
256
|
+
3. 🎲 **Seed** — one number. A persistent directory pins it; an explicit `seed=`
|
|
257
|
+
reproduces a run exactly.
|
|
258
|
+
4. 🧍 **Profile** — user agent, client hints, platform, screen, viewport, GPU
|
|
259
|
+
strings, cores and memory, all derived from that seed **and from the real
|
|
260
|
+
binary's version**, so a 157 build never claims to be 139.
|
|
261
|
+
5. 🚩 **Flags** — built by one merge function that de-duplicates by flag key, so a
|
|
262
|
+
flag can never be present on one launch path and missing on another.
|
|
263
|
+
6. 🛰️ **Relay** — a loopback HTTP relay holds the proxy credentials and attaches
|
|
264
|
+
`Proxy-Authorization` upstream, so Chromium never sees a `407`.
|
|
265
|
+
7. 🖥️ **Launch** — `--headless=new`, the real browser without a window, never the
|
|
266
|
+
stripped headless shell.
|
|
267
|
+
8. 🧪 **Inject** — the fingerprint layer, applied to every realm.
|
|
268
|
+
9. 🖱️ **Humanize** — Playwright's input primitives are replaced.
|
|
269
|
+
|
|
270
|
+
---
|
|
271
|
+
|
|
272
|
+
## 🕵️ The fingerprint layer
|
|
273
|
+
|
|
274
|
+
Each rule closes a specific, cheap detection:
|
|
275
|
+
|
|
276
|
+
- 🧬 **Prototype, never instance** — every patch is a getter on
|
|
277
|
+
`Navigator.prototype`, so `Object.getOwnPropertyNames(navigator)` stays empty.
|
|
278
|
+
An own property on the instance is visible in one line of JavaScript.
|
|
279
|
+
- 🎭 **Native source** — a single patched `Function.prototype.toString` reports
|
|
280
|
+
`[native code]` for every replacement, and it **delegates to the parent realm**,
|
|
281
|
+
which closes the classic escape of borrowing a fresh iframe's `toString` and
|
|
282
|
+
calling it on a patched getter.
|
|
283
|
+
- 🖼️ **Every realm** — dynamic `about:blank` iframes, `srcdoc`, nested frames and
|
|
284
|
+
`window.frames["name"]`, each verified separately.
|
|
285
|
+
- 🎨 **Deterministic noise** — canvas and audio noise derives from the session
|
|
286
|
+
seed, so reading the same canvas twice returns the same bytes, exactly as a
|
|
287
|
+
real GPU does. Random-per-call noise is itself a signal.
|
|
288
|
+
- 🧩 **Fill gaps, do not invent** — on a real Chrome the `chrome` object is left
|
|
289
|
+
alone, `chrome.runtime` **included**: it genuinely does not exist on a normal
|
|
290
|
+
page, and a stub that adds it stands out more than no stub at all.
|
|
291
|
+
- 📋 **Headers agree with JavaScript** — client hints are set through CDP and that
|
|
292
|
+
session is kept alive for the page's lifetime, because detaching reverts the
|
|
293
|
+
override and leaves `Sec-CH-UA` disagreeing with `navigator.userAgentData`. The
|
|
294
|
+
brand list is copied from a real Chrome on the wire, GREASE entry and order
|
|
295
|
+
included.
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
## 🖱️ The human layer
|
|
300
|
+
|
|
301
|
+
<p align="center"><img src="docs/images/pointer-path.svg" alt="a real human_click path" width="100%"></p>
|
|
302
|
+
|
|
303
|
+
Timings are not invented. Inter-key intervals and hold times follow the
|
|
304
|
+
distributions reported for 136M keystrokes (Dhakal et al., CHI 2018); pointing
|
|
305
|
+
movements follow Fitts' law with the fast-stroke-plus-correction structure from
|
|
306
|
+
the sub-movement literature — which is why a long move sometimes misses and
|
|
307
|
+
lands with a short second stroke.
|
|
308
|
+
|
|
309
|
+
<p align="center"><img src="docs/images/wheel.svg" alt="wheel notches" width="100%"></p>
|
|
310
|
+
|
|
311
|
+
```python
|
|
312
|
+
page.human_click("#go")
|
|
313
|
+
page.human_type("#email", "a@b.com")
|
|
314
|
+
page.human_fill("#email", "a@b.com")
|
|
315
|
+
page.human_scroll(600)
|
|
316
|
+
page.human_move(400, 300)
|
|
317
|
+
page.human_idle(1.5)
|
|
318
|
+
page.human
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
Tuning:
|
|
322
|
+
|
|
323
|
+
```python
|
|
324
|
+
launch(humanize="careful")
|
|
325
|
+
launch(humanize="brisk")
|
|
326
|
+
launch(humanize={"typing_delay": 90})
|
|
327
|
+
launch(humanize=HumanConfig(mistype_chance=0.0))
|
|
328
|
+
launch(humanize=False)
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
Failures are specific — `ElementNotVisibleError`, `ElementNotStableError`,
|
|
332
|
+
`ElementNotEditableError`, `ElementNotReceivingEventsError` — so a failing
|
|
333
|
+
selector tells you *why*, not just that it timed out.
|
|
334
|
+
|
|
335
|
+
---
|
|
336
|
+
|
|
337
|
+
## 🌍 Proxy, geo and the relay
|
|
338
|
+
|
|
339
|
+
```python
|
|
340
|
+
launch(proxy="http://user:pass@gateway:8080")
|
|
341
|
+
launch(proxy={"server": "http://gw:8080", "username": "u", "password": "p"})
|
|
342
|
+
launch(proxy=PROXY, geoip="strict")
|
|
343
|
+
launch(proxy=PROXY, geoip=False, timezone_id="Asia/Bangkok", locale="th-TH")
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
- 🛰️ The relay listens on `127.0.0.1` with **no** authentication of its own and
|
|
347
|
+
attaches `Proxy-Authorization` upstream. Chromium gets a plain
|
|
348
|
+
`--proxy-server=http://127.0.0.1:<port>`, so no password reaches a child
|
|
349
|
+
process and no tunnel starts with a `407`.
|
|
350
|
+
- 🌐 The geo lookup runs **through the proxy**, because the gateway and the exit
|
|
351
|
+
are frequently different hosts in different countries. It sends no identifiers
|
|
352
|
+
and can be turned off entirely.
|
|
353
|
+
- 🧦 SOCKS proxies skip both — Chromium cannot authenticate SOCKS on the command
|
|
354
|
+
line either, and `urllib` cannot route the lookup. Pass `timezone_id` and
|
|
355
|
+
`locale` yourself there.
|
|
356
|
+
|
|
357
|
+
---
|
|
358
|
+
|
|
359
|
+
## ✅ Proving it
|
|
360
|
+
|
|
361
|
+
<p align="center"><img src="docs/images/selfcheck.svg" alt="grewbrowser check output" width="100%"></p>
|
|
362
|
+
|
|
363
|
+
```bash
|
|
364
|
+
grewbrowser check
|
|
365
|
+
grewbrowser check --mode headless
|
|
366
|
+
grewbrowser check --url https://your-own-site.example
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
Probes are split into two tiers, which matters the moment you run this on a
|
|
370
|
+
server:
|
|
371
|
+
|
|
372
|
+
- **coherence** — everything grewbrowser controls and must always get right.
|
|
373
|
+
A failure here is a defect, and it is what the exit code reflects.
|
|
374
|
+
- **this machine** — properties of the host that no patching can create: a
|
|
375
|
+
hardware GPU, installed fonts, speech-synthesis voices. A GPU-less CI box
|
|
376
|
+
legitimately fails these, and that is reported, not treated as a bug.
|
|
377
|
+
|
|
378
|
+
Each tier is printed with a `<- differs` marker on any check where headless and
|
|
379
|
+
headful disagree. Everything runs against a page served from loopback, so there
|
|
380
|
+
is no network call and no third-party scoring service — loopback is also a
|
|
381
|
+
*secure context*, which `about:blank` is not, and without it `mediaDevices` and
|
|
382
|
+
WebGPU would look missing when they are merely unavailable on an insecure origin.
|
|
383
|
+
|
|
384
|
+
It covers the identity, window geometry, the GPU, canvas and audio noise,
|
|
385
|
+
permissions, media devices, WebRTC candidate types, all four iframe realm types
|
|
386
|
+
plus the cross-realm `toString` escape, the known headless tells (fonts,
|
|
387
|
+
speech-synthesis voices, `navigator.connection`, battery API, hover and pointer
|
|
388
|
+
media queries, screen orientation, shader precision), the agreement between what
|
|
389
|
+
the browser **sent** and what JavaScript **says**, and the input layer.
|
|
390
|
+
|
|
391
|
+
`run_checks()` is importable if you would rather assert on it in your own tests.
|
|
392
|
+
|
|
393
|
+
> ⚠️ This measures **self-consistency**, not a bot-detection score. It is not a
|
|
394
|
+
> CAPTCHA bypass and makes no claim about any anti-bot vendor.
|
|
395
|
+
|
|
396
|
+
---
|
|
397
|
+
|
|
398
|
+
## 🧩 Extensions
|
|
399
|
+
|
|
400
|
+
```python
|
|
401
|
+
launch_persistent_context("./profiles/x", extension_paths=["./my_extension"])
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
Extensions need a **persistent context**, not a headful one. `--headless=new`
|
|
405
|
+
loads them fine, but `launch()` builds a non-persistent context, which Chromium
|
|
406
|
+
treats as incognito and does not run content scripts in — while the service
|
|
407
|
+
worker still starts, so it *looks* loaded when it is not. grewbrowser warns if
|
|
408
|
+
you use the wrong one.
|
|
409
|
+
|
|
410
|
+
`examples/demo_extension/` is a two-line extension that marks each page, so you
|
|
411
|
+
can prove loading works before trusting it with a real one.
|
|
412
|
+
|
|
413
|
+
---
|
|
414
|
+
|
|
415
|
+
## 🖥️ Platforms
|
|
416
|
+
|
|
417
|
+
| | Windows | Linux | macOS |
|
|
418
|
+
|---|:---:|:---:|:---:|
|
|
419
|
+
| Own Chromium download | ✅ | ✅ | ✅ |
|
|
420
|
+
| ANGLE backend | D3D11 | GL | Metal |
|
|
421
|
+
| Wheel notch | 100px | 120px | 40px |
|
|
422
|
+
| Process-tree teardown | `taskkill /T` | `killpg` | `killpg` |
|
|
423
|
+
| CI, offline + live | ✅ | ✅ *(Xvfb)* | ✅ |
|
|
424
|
+
|
|
425
|
+
Linux arm64 has no published Chrome for Testing build — use `executable_path=`
|
|
426
|
+
with your own. Headful on Linux needs `DISPLAY` or `WAYLAND_DISPLAY`;
|
|
427
|
+
grewbrowser warns when neither is set.
|
|
428
|
+
|
|
429
|
+
---
|
|
430
|
+
|
|
431
|
+
## ⚙️ Reference
|
|
432
|
+
|
|
433
|
+
### Launch arguments
|
|
434
|
+
|
|
435
|
+
| Argument | Default | Meaning |
|
|
436
|
+
|---|---|---|
|
|
437
|
+
| `headless` | `True` | `--headless=new`, the real browser without a window |
|
|
438
|
+
| `proxy` | `None` | URL string or Playwright mapping |
|
|
439
|
+
| `geoip` | `True` | `True` warns on failure, `"strict"` raises, `False` skips |
|
|
440
|
+
| `humanize` | `True` | preset name, `HumanConfig`, mapping of overrides, or `False` |
|
|
441
|
+
| `seed` | random | same seed, same identity |
|
|
442
|
+
| `os_family` | host's | `"windows"` / `"darwin"` / `"linux"` |
|
|
443
|
+
| `locale`, `timezone_id` | from geo | override the lookup |
|
|
444
|
+
| `viewport`, `screen` | derived | layout viewport and screen resolution |
|
|
445
|
+
| `channel` | `"auto"` | see the channel table |
|
|
446
|
+
| `executable_path` | `None` | your own binary |
|
|
447
|
+
| `args` | `None` | extra flags; these **win** over the stealth defaults |
|
|
448
|
+
| `stealth_args` | `True` | include the stealth flag list at all |
|
|
449
|
+
| `inject_fingerprint` | auto | off automatically for a patched Chromium |
|
|
450
|
+
| `relay` | `True` | move proxy credentials to the loopback relay |
|
|
451
|
+
| `native_viewport` | `True` | viewport follows the real window instead of being emulated |
|
|
452
|
+
| `extension_paths` | `None` | unpacked extensions (persistent contexts) |
|
|
453
|
+
| `permissions` | `None` | granted up front |
|
|
454
|
+
| `**context_kwargs` | — | passed straight to `new_context` |
|
|
455
|
+
|
|
456
|
+
### Environment
|
|
457
|
+
|
|
458
|
+
| Variable | Effect |
|
|
459
|
+
|---|---|
|
|
460
|
+
| `GREWBROWSER_BINARY_PATH` | use this executable, above everything else |
|
|
461
|
+
| `GREWBROWSER_AUTO_DOWNLOAD` | `0` stops `channel="auto"` from ever downloading |
|
|
462
|
+
| `GREWBROWSER_CACHE_DIR` | where managed builds live (default `~/.grewbrowser`) |
|
|
463
|
+
| `GREWBROWSER_PATCHED_BINARY` | `1` marks the binary as a fingerprint-patched build |
|
|
464
|
+
| `GREWBROWSER_GEOIP_TIMEOUT` | seconds per geo provider (default 8) |
|
|
465
|
+
|
|
466
|
+
### Modules
|
|
467
|
+
|
|
468
|
+
| Module | Responsibility |
|
|
469
|
+
|---|---|
|
|
470
|
+
| `core.profile` | one seed in, one coherent identity out |
|
|
471
|
+
| `core.flags` | `build_stealth_args`, the flag merge, seed pinning |
|
|
472
|
+
| `core.binary_manager` | discovery: explicit path → env → own build → Playwright → system |
|
|
473
|
+
| `core.download` | fetching and caching grewbrowser's own Chromium |
|
|
474
|
+
| `fingerprint.scripts` | the injected payload |
|
|
475
|
+
| `fingerprint.applier` | context options and the CDP client-hint override |
|
|
476
|
+
| `behavior.curves` / `.keyboard` | pure motion and keystroke planning |
|
|
477
|
+
| `behavior.world` | CDP isolated worlds for actionability probes |
|
|
478
|
+
| `behavior.human` | the engines and the `human_*` page methods |
|
|
479
|
+
| `network.proxy` / `.geoip` / `.relay` | proxy parsing, exit lookup, credential relay |
|
|
480
|
+
| `session.pool` / `.teardown` | concurrency limits and guaranteed cleanup |
|
|
481
|
+
| `diagnostics` | the self-check probes |
|
|
482
|
+
|
|
483
|
+
Every module stands alone: `build_stealth_args` without launching, `resolve_geo`
|
|
484
|
+
without a browser, `humanize_page(page)` on a page you opened yourself.
|
|
485
|
+
|
|
486
|
+
---
|
|
487
|
+
|
|
488
|
+
## 🧪 Development
|
|
489
|
+
|
|
490
|
+
```bash
|
|
491
|
+
git clone https://github.com/x2super/grewbrowser
|
|
492
|
+
cd grewbrowser
|
|
493
|
+
pip install -e ".[dev]"
|
|
494
|
+
|
|
495
|
+
pytest tests/test_offline.py -q # no browser needed
|
|
496
|
+
grewbrowser install
|
|
497
|
+
pytest tests/test_live.py -q # launches a real browser
|
|
498
|
+
grewbrowser check
|
|
499
|
+
```
|
|
500
|
+
|
|
501
|
+
CI runs the offline suite on Windows, Linux and macOS across Python 3.10 and
|
|
502
|
+
3.13, and the live suite plus the self-check on all three — Linux under Xvfb.
|
|
503
|
+
|
|
504
|
+
---
|
|
505
|
+
|
|
506
|
+
## 🚧 Honest limits
|
|
507
|
+
|
|
508
|
+
- 🧪 **The fingerprint layer is JavaScript.** Careful JavaScript, but a Chromium
|
|
509
|
+
patched in C++ is strictly stronger. Point `executable_path` at one and
|
|
510
|
+
grewbrowser detects it (or set `GREWBROWSER_PATCHED_BINARY=1`) and skips
|
|
511
|
+
injection rather than overwriting correct native values.
|
|
512
|
+
- 🎭 **Spoofing a foreign OS is supported but not advised.** Fonts and the real
|
|
513
|
+
GPU come from the host and will contradict the claim.
|
|
514
|
+
- 🌐 **The geo lookup is a third-party request.** It carries no identifiers, and
|
|
515
|
+
`geoip=False` turns it off.
|
|
516
|
+
- 🧦 **SOCKS** skips the relay and the geo lookup.
|
|
517
|
+
- 🔒 **Chrome for Testing archives carry no publisher signature.** TLS to Google's
|
|
518
|
+
host is the trust anchor for the download — the same anchor `playwright
|
|
519
|
+
install` relies on. The archive's SHA-256 is recorded locally so you can see
|
|
520
|
+
what you got.
|
|
521
|
+
- 🚫 **No CAPTCHA solving and no authentication bypass.** Not included, not
|
|
522
|
+
planned.
|
|
523
|
+
|
|
524
|
+
Built for testing your own services, scraping with permission, and research into
|
|
525
|
+
browser fingerprinting.
|
|
526
|
+
|
|
527
|
+
---
|
|
528
|
+
|
|
529
|
+
<p align="center">
|
|
530
|
+
<sub>MIT © <a href="https://github.com/x2super">x2super</a> · <a href="CHANGELOG.md">changelog</a></sub>
|
|
531
|
+
</p>
|