unrealon 1.1.4__py3-none-any.whl → 1.1.6__py3-none-any.whl

{unrealon-1.1.4.dist-info → unrealon-1.1.6.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: unrealon
-Version: 1.1.4
+Version: 1.1.6
 Summary: 🚀 Revolutionary web scraping platform with unbreakable stealth, AI-powered extraction, and zero-config setup. Build enterprise parsers in minutes, not months!
 Project-URL: Homepage, https://github.com/unrealos/unrealon-rpc
 Project-URL: Documentation, https://unrealon-rpc.readthedocs.io
@@ -36,12 +36,14 @@ Classifier: Topic :: System :: Distributed Computing
 Classifier: Typing :: Typed
 Requires-Python: <4.0,>=3.10
 Requires-Dist: aiohttp>=3.9.0
+Requires-Dist: aioipfs<0.8.0,>=0.7.1
 Requires-Dist: asyncio-mqtt>=0.16.0
 Requires-Dist: beautifulsoup4>=4.13.4
 Requires-Dist: click>=8.2.0
 Requires-Dist: httpx>=0.26.0
 Requires-Dist: ipfshttpclient>=0.8.0a2
 Requires-Dist: lxml>=6.0.0
+Requires-Dist: msgpack<2.0.0,>=1.1.1
 Requires-Dist: playwright-stealth>=2.0.0
 Requires-Dist: playwright>=1.54.0
 Requires-Dist: pydantic-yaml<2.0.0,>=1.6.0
@@ -49,9 +51,11 @@ Requires-Dist: pydantic<3.0,>=2.11
 Requires-Dist: python-dateutil>=2.8
 Requires-Dist: python-dotenv>=1.0.0
 Requires-Dist: pyyaml>=6.0
+Requires-Dist: questionary<3.0.0,>=2.1.0
 Requires-Dist: redis>=5.0.0
 Requires-Dist: rich>=13.0.0
 Requires-Dist: tomlkit>=0.13.0
+Requires-Dist: watchdog<7.0.0,>=6.0.0
 Requires-Dist: websockets>=12.0
 Provides-Extra: dev
 Requires-Dist: bandit>=1.7.0; extra == 'dev'
@@ -180,122 +184,85 @@ await parser.start_daemon()
 
 ### 🎯 **Developer's Perspective - Simple & Clean**
 
-```mermaid
-graph TD
-    A["💻 Your Parser Code<br/>🐍 Python Script<br/><br/>class MyParser(ParserManager):<br/>    async def parse_products(url):<br/>        return await self.extract_with_ai(url)"]
-    
-    B["🚀 Built-in Browser<br/>🎭 Playwright + Stealth<br/><br/>✅ Anti-Detection<br/>✅ Proxy Rotation<br/>✅ CAPTCHA Solving"]
-    
-    C["🌐 Target Websites<br/><br/>🛒 E-commerce Sites<br/>📰 News Portals<br/>📱 Social Media<br/>🌍 Any Website"]
-    
-    D["📊 UnrealOn Dashboard<br/><br/>📈 Real-time Monitoring<br/>📋 Logs & Analytics<br/>⚙️ Task Management<br/>💾 Data Storage"]
-    
-    A --> B
-    B --> C
-    A -.->|"🔄 Automatic Sync<br/>📊 Metrics & Logs<br/>📦 Parsed Data"| D
-    
-    style A fill:#e3f2fd,stroke:#1976d2,stroke-width:3px
-    style B fill:#f3e5f5,stroke:#7b1fa2,stroke-width:3px
-    style C fill:#fff3e0,stroke:#f57c00,stroke-width:3px
-    style D fill:#e8f5e8,stroke:#388e3c,stroke-width:3px
-```
+**Architecture Overview - Developer's Perspective:**
+
+- **💻 Your Parser Code (Python Script)**
+  - Simple class extending ParserManager
+  - Focus on business logic only
+  - Example: `async def parse_products(url): return await self.extract_with_ai(url)`
+
+- **🚀 Built-in Browser (Playwright + Stealth)**
+  - ✅ Anti-Detection
+  - ✅ Proxy Rotation
+  - ✅ CAPTCHA Solving
+
+- **🌐 Target Websites**
+  - 🛒 E-commerce Sites
+  - 📰 News Portals
+  - 📱 Social Media
+  - 🌍 Any Website
+
+- **📊 UnrealOn Dashboard**
+  - 📈 Real-time Monitoring
+  - 📋 Logs & Analytics
+  - ⚙️ Task Management
+  - 💾 Data Storage
+
+**Flow:** Your code → Built-in Browser → Target Websites
+**Automatic Sync:** Your code ⟷ UnrealOn Dashboard (metrics, logs, parsed data)
 
 ### 🔄 **Two Operation Modes**
 
 #### 🔧 **Standalone Mode** (Local Development)
-```mermaid
-graph TD
-    A["💻 Your Parser<br/>🐍 Local Python Script"]
-    B["🚀 Built-in Browser<br/>🎭 Stealth Enabled"]
-    C["🌐 Target Website<br/>🛒 E-commerce / 📰 News"]
-    D["💾 Local Results<br/>📄 JSON / CSV / Database"]
-    
-    A --> B
-    B --> C
-    B --> D
-    
-    style A fill:#e3f2fd,stroke:#1976d2,stroke-width:3px
-    style B fill:#f3e5f5,stroke:#7b1fa2,stroke-width:3px
-    style C fill:#fff3e0,stroke:#f57c00,stroke-width:3px
-    style D fill:#e8f5e8,stroke:#388e3c,stroke-width:3px
-```
+**Standalone Mode Flow:**
+
+- 💻 Your Parser (Local Python Script)
+- 🚀 Built-in Browser with Stealth Enabled
+- 🌐 Target Website (E-commerce/News)
+- 💾 Local Results (JSON/CSV/Database)
+
+**Process:** Your Parser → Browser → Target Website → Local Results
 
 #### 🚀 **Dashboard Mode** (Production)
-```mermaid
-graph TD
-    A["💻 Your Parser<br/>🐍 Production Script"]
-    B["🚀 Built-in Browser<br/>🎭 Enterprise Stealth"]
-    C["🌐 Target Website<br/>🛒 E-commerce / 📰 News"]
-    D["📊 UnrealOn Dashboard<br/>☁️ Cloud Platform"]
-    E["👥 Team Collaboration<br/>🔐 Role Management"]
-    F["📈 Analytics & Reports<br/>📊 Business Intelligence"]
-    G["📤 Data Export<br/>🔗 API / Webhooks"]
-    
-    A --> B
-    B --> C
-    A --> D
-    D --> E
-    D --> F
-    D --> G
-    
-    style A fill:#e3f2fd,stroke:#1976d2,stroke-width:3px
-    style B fill:#f3e5f5,stroke:#7b1fa2,stroke-width:3px
-    style C fill:#fff3e0,stroke:#f57c00,stroke-width:3px
-    style D fill:#e8f5e8,stroke:#388e3c,stroke-width:3px
-    style E fill:#e8f5e8,stroke:#388e3c,stroke-width:2px
-    style F fill:#e8f5e8,stroke:#388e3c,stroke-width:2px
-    style G fill:#e8f5e8,stroke:#388e3c,stroke-width:2px
-```
+**Dashboard Mode Flow:**
+
+- 💻 Your Parser (Production Script)
+- 🚀 Built-in Browser with Enterprise Stealth
+- 🌐 Target Website (E-commerce/News)
+- 📊 UnrealOn Dashboard (Cloud Platform)
+  - 👥 Team Collaboration & Role Management
+  - 📈 Analytics & Business Intelligence Reports
+  - 📤 Data Export via API/Webhooks
+
+**Process:** 
+- Parser → Browser → Target Website
+- Parser → Dashboard → Team/Analytics/Export
 
 ### 🎯 **What You Focus On vs What UnrealOn Handles**
 
-```mermaid
-graph TD
-    subgraph "🎯 Your Focus - Business Logic Only"
-        A1["🎯 Define Target URLs<br/>📝 urls = ['amazon.com', 'ebay.com']"]
-        A2["🔍 Specify Data to Extract<br/>📋 'Extract title, price, rating'"]
-        A3["📊 Handle Results<br/>💾 Save to database/API"]
-        A4["⏰ Schedule Tasks<br/>🕐 Run every hour/daily"]
-    end
-    
-    A1 --> A2
-    A2 --> A3
-    A3 --> A4
-    
-    subgraph "🚀 UnrealOn Handles All Infrastructure"
-        B1["🌐 Browser Management<br/>🎭 Playwright + Chrome"]
-        B2["🛡️ Stealth & Anti-Detection<br/>🔒 Fingerprint Spoofing"]
-        B3["🔄 Proxy Rotation<br/>🌍 Global IP Pool"]
-        B4["🤖 CAPTCHA Solving<br/>🧩 Automatic Resolution"]
-        B5["⚠️ Error Handling<br/>🔄 Retry Logic"]
-        B6["📈 Logging & Monitoring<br/>📊 Real-time Metrics"]
-        B7["💾 Data Storage<br/>☁️ Cloud Database"]
-        B8["⚡ Performance Optimization<br/>🚀 Auto-scaling"]
-    end
-    
-    A1 -.->|"Automatically Triggers"| B1
-    A2 -.->|"Automatically Triggers"| B2
-    A2 -.->|"Automatically Triggers"| B3
-    A3 -.->|"Automatically Triggers"| B4
-    A3 -.->|"Automatically Triggers"| B5
-    A4 -.->|"Automatically Triggers"| B6
-    A4 -.->|"Automatically Triggers"| B7
-    A4 -.->|"Automatically Triggers"| B8
-    
-    style A1 fill:#e3f2fd,stroke:#1976d2,stroke-width:3px
-    style A2 fill:#e3f2fd,stroke:#1976d2,stroke-width:3px
-    style A3 fill:#e3f2fd,stroke:#1976d2,stroke-width:3px
-    style A4 fill:#e3f2fd,stroke:#1976d2,stroke-width:3px
-    
-    style B1 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
-    style B2 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
-    style B3 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
-    style B4 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
-    style B5 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
-    style B6 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
-    style B7 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
-    style B8 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
-```
+**What You Focus On vs What UnrealOn Handles:**
+
+**🎯 Your Focus - Business Logic Only:**
+1. 🎯 Define Target URLs
+   - Example: `urls = ['amazon.com', 'ebay.com']`
+2. 🔍 Specify Data to Extract
+   - Example: `'Extract title, price, rating'`
+3. 📊 Handle Results
+   - Save to database/API
+4. ⏰ Schedule Tasks
+   - Run every hour/daily
+
+**🚀 UnrealOn Handles All Infrastructure:**
+1. 🌐 Browser Management (Playwright + Chrome)
+2. 🛡️ Stealth & Anti-Detection (Fingerprint Spoofing)
+3. 🔄 Proxy Rotation (Global IP Pool)
+4. 🤖 CAPTCHA Solving (Automatic Resolution)
+5. ⚠️ Error Handling (Retry Logic)
+6. 📈 Logging & Monitoring (Real-time Metrics)
+7. 💾 Data Storage (Cloud Database)
+8. ⚡ Performance Optimization (Auto-scaling)
+
+**Each of your actions automatically triggers the corresponding infrastructure components.**
 
 **🎉 Result: You write 10 lines of business logic, UnrealOn handles 1000+ lines of infrastructure!**
 
@@ -384,15 +351,15 @@ await parser.start_daemon(schedule="1h")
 - **CAPTCHA Solving** - Automatic CAPTCHA resolution
 - **Behavioral Patterns** - User action simulation
 
-### Stealth Levels:
+### Stealth Features:
 ```python
-# Configure stealth level
-parser = ParserManager(stealth_level="maximum")  # minimal | balanced | maximum
+# Stealth is always enabled by default
+parser = ParserManager()  # 🔥 STEALTH ALWAYS ON!
 ```
 
-- **Minimal** - Basic protection (fast)
-- **Balanced** - Optimal balance (recommended)
-- **Maximum** - Maximum protection (slower but bulletproof)
+- **Webdriver Detection Prevention** - Hides automation signals
+- **Browser Fingerprint Randomization** - Unique fingerprints
+- **JavaScript API Modifications** - Prevents detection
 
 ---
 

{unrealon-1.1.4.dist-info → unrealon-1.1.6.dist-info}/RECORD

@@ -1,27 +1,28 @@
-unrealon/__init__.py,sha256=EEqg3RwVJZpXddYS8ANBYoOLa2hE2B-JDY8VUxHdvno,922
 unrealon_browser/README.md,sha256=9pP6RrfMGHtdT5uDLFAUB1e4nNGzZudXViEo1940gKw,396
-unrealon_browser/__init__.py,sha256=kvp3aZApfvP__iAnFzsHI25XKS9IWnt9znUI9YZbgAk,1428
+unrealon_browser/__init__.py,sha256=Bk1jERQP3CpwfExd3UmydNRIZcEIylQaEfP430p9Vjg,1502
 unrealon_browser/cli/__init__.py,sha256=b3r88oeCYsqZF8EU8EZXP9v54Q8cIimN7UmxJsXcB84,264
 unrealon_browser/cli/browser_cli.py,sha256=SRRCGbNXaEg1ZN04-jPo9GOWcP2b6Bkalu6PYVOOt5k,8342
 unrealon_browser/cli/cookies_cli.py,sha256=yhZvGrg8bknlH4zlySdi8ue-25Ue-1rI_u1G06OIMg4,13304
-unrealon_browser/cli/interactive_mode.py,sha256=iYt9PNaIBhNjZ9aUa1ZxeeneV2u1VTSkYQ6rsyio-o8,11730
+unrealon_browser/cli/interactive_mode.py,sha256=KgdPPa5rZEEApc92JQz346WMESAFDBHERpz_dxeZgKk,11885
 unrealon_browser/cli/main.py,sha256=XCYcTxJUqaz320KCU_JPKizYMk6bdljb8Boyok3uO-4,1353
 unrealon_browser/core/__init__.py,sha256=uVL_t4sZelUzflWPdgrwoXGnAkSV1WNQ98-eu0QB2eM,151
-unrealon_browser/core/browser_manager.py,sha256=uCr0VYjLCep9Nr1b60wtUDEqLEdvqWae0sAZYzH61vo,26334
+unrealon_browser/core/browser_manager.py,sha256=-m6ZeuRTUvI7t7vZ20xK5klR6px9eeswSv1z6eGkD2k,29056
 unrealon_browser/dto/__init__.py,sha256=p9mG2QwnXEdHUHYK67vGD6aameM8RkiVATzz8y0u5EE,1206
-unrealon_browser/dto/models/config.py,sha256=XSUkWyam-LWRbqlx1qE9GPosaJZo1ZsW-o_ARyaQ--Q,1019
+unrealon_browser/dto/models/config.py,sha256=3O5nk6vFw6Kxgu941lJIg9dGAUzXacQn0yvxC1mGEa4,1585
 unrealon_browser/dto/models/core.py,sha256=HvbwYG27rmmWtp401uws7lfalN_9QPad0M6ceCiN5iQ,2741
 unrealon_browser/dto/models/dataclasses.py,sha256=zqhJVyzp4CvtuTBsZwm6n6TodVWrZf9gkdDG-0_tgeA,2571
 unrealon_browser/dto/models/detection.py,sha256=ma9ZNIjPR7HnjqZaAj6ZoskiewPFiSn_FgFXSkgiQc8,2715
 unrealon_browser/dto/models/enums.py,sha256=Q4WzHdfSKf7dhKyX00i_Pvl2U8w3lBsxOYfSIoaQY3Q,1219
 unrealon_browser/dto/models/statistics.py,sha256=aIzJNV5r23VBxjhEoja4tXwI1Z7_UCw5zOaxuPya2E8,2728
-unrealon_browser/managers/__init__.py,sha256=JuH9FW_kTzVv71jCDp6wOT4SXT6HGSBpyNAb4tD7-ck,456
+unrealon_browser/managers/__init__.py,sha256=ec5gvXFv2iLJD9rVDZkKF_yszqOSxVnl9m8AWEVmI8c,527
 unrealon_browser/managers/captcha.py,sha256=KGBO7sfq9XusAlcPByUFdIg-v6rlruzS2oHx-Zx28wo,21453
 unrealon_browser/managers/cookies.py,sha256=r4VVnKLXH82vhU7qgtY-dF7KPf0Ie3QxGD3FEi6geFA,15085
-unrealon_browser/managers/logger_bridge.py,sha256=GSxFSE7I596NQ8zFvWtf5IF9YADexHwvBP2-5FK8wrY,11066
+unrealon_browser/managers/logger_bridge.py,sha256=wWf13oizI3fBXbTvE_ghjhWbb2q8xp9gsTTUm9UNmJg,11098
+unrealon_browser/managers/page_wait_manager.py,sha256=UyZqiSfkjzahrxp9x1odXFIT_sFhZGvdECxWuIMCVBY,7876
 unrealon_browser/managers/profile.py,sha256=HjddlSeUry_65WPtF8CMkT7cfJ6X3Jap9kJaaZpwtAA,18956
 unrealon_browser/managers/stealth.py,sha256=-ge67IAueO6zRa0SSffi8Fpd2mhG_2IyRvZADzxJbfk,14651
-unrealon_driver/__init__.py,sha256=S0s4m-MpDXGVuQVfcqZ-PccFixGwKgww065LfnK-4fE,1532
+unrealon_driver/README.md,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+unrealon_driver/__init__.py,sha256=GZq15ye175N4MchCxMklfDzbIokLcmIZC92fMPgx4ls,1600
 unrealon_driver/exceptions.py,sha256=5b7ndK-UROfVvON3qf9YVXK4tq8PZzgtBSHTkiA1Oos,437
 unrealon_driver/html_analyzer/__init__.py,sha256=sDrhKYJ9I3OdgbL60w92WxfiHD_Jhcw0vgV2XYkKQwg,1007
 unrealon_driver/html_analyzer/cleaner.py,sha256=77umjQojIl9BMSFpe4j6x3_5TMQh7Jk5GXUhuempu3k,25198
@@ -47,8 +48,8 @@ unrealon_driver/websocket/__init__.py,sha256=qwOOIKEcwvP_0GYW6fTAmNdiUMHaotG_IL5
 unrealon_driver/websocket/client.py,sha256=NvfbeDe_uDfLniS2q6Qzd5p4UnhSqvdSzUaUXNMtzLw,9179
 unrealon_driver/websocket/config.py,sha256=ez6EjAOPjDMoQeXHEkulT90o2vpeC34eShgNFLX2g1E,6500
 unrealon_driver/websocket/manager.py,sha256=8RAq3NuBFfV2y0ThAZVD7IhO_t7OAJV9phvnZooaLlc,2779
-unrealon-1.1.4.dist-info/METADATA,sha256=xwBrCFv9aUhu3QQYBK5irSMuZnP--zleZQZBB3zhip4,21402
-unrealon-1.1.4.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
-unrealon-1.1.4.dist-info/entry_points.txt,sha256=s-tuFkQDg3TQ3U5k0yzt6oWGUhTLum0BLdCQEIDnpiU,431
-unrealon-1.1.4.dist-info/licenses/LICENSE,sha256=uTZpktXKUsE0IzS5RdSV398HHI74ssbGKTdCbv7U9l0,1070
-unrealon-1.1.4.dist-info/RECORD,,
+unrealon-1.1.6.dist-info/METADATA,sha256=ZR5NN3JCehzeweWQhnVYteEhVeJdrviGCQ_7SzpbFOc,19458
+unrealon-1.1.6.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+unrealon-1.1.6.dist-info/entry_points.txt,sha256=s-tuFkQDg3TQ3U5k0yzt6oWGUhTLum0BLdCQEIDnpiU,431
+unrealon-1.1.6.dist-info/licenses/LICENSE,sha256=uTZpktXKUsE0IzS5RdSV398HHI74ssbGKTdCbv7U9l0,1070
+unrealon-1.1.6.dist-info/RECORD,,

unrealon_browser/__init__.py

@@ -5,8 +5,12 @@ Enterprise-grade browser automation with stealth capabilities and proxy integrat
 Based on proven patterns from unrealparser with modular architecture.
 """
 
-from unrealon import VersionInfo
-__version__ = VersionInfo().version
+try:
+    from importlib.metadata import version
+
+    __version__ = version("unrealon-browser")
+except Exception:
+    __version__ = "1.0.0-dev"
 
 # Core browser management
 from .core import BrowserManager

unrealon_browser/cli/interactive_mode.py

@@ -97,9 +97,10 @@ async def _interactive_browser_launch(parser: str, verbose: bool) -> None:
     
     headless = questionary.confirm("Run in headless mode?", default=False).ask()
     
-    stealth_level = questionary.select(
-        "Select stealth level:",
-        choices=["disabled", "basic", "advanced"]
+    # 🔥 STEALTH ALWAYS ON - NO CONFIG NEEDED!
+    stealth_info = questionary.select(
+        "Stealth is always enabled. Select stealth verification:",
+        choices=["None", "Test on bot.sannysoft.com", "Test on fingerprint.com"]
     ).ask()
     
     url = questionary.text(
@@ -114,7 +115,8 @@ async def _interactive_browser_launch(parser: str, verbose: bool) -> None:
 Parser: [cyan]{parser}[/cyan]
 Browser: [green]{browser_type}[/green]
 Mode: [yellow]{'Headless' if headless else 'GUI'}[/yellow]
-Stealth: [magenta]{stealth_level}[/magenta]
+Stealth: [magenta]ALWAYS ON[/magenta]
+Verification: [magenta]{stealth_info}[/magenta]
 Target URL: [blue]{url}[/blue]
 """
     

unrealon_browser/core/browser_manager.py

@@ -29,6 +29,7 @@ from unrealon_browser.managers import (
     CookieManager,
     CaptchaDetector,
     create_browser_logger_bridge,
+    PageWaitManager,
 )
 
 
@@ -62,6 +63,7 @@ class BrowserManager:
         self.cookie_manager = None
         self.captcha_manager = CaptchaDetector()
         self.logger_bridge = create_browser_logger_bridge(session_id=self._generate_session_id(), parser_id=self.parser_id, enable_console=True)  # Use resolved parser_id
+        self.page_wait = PageWaitManager(None, self.logger_bridge)
 
         # Signal handlers for graceful shutdown
         self._setup_signal_handlers()
@@ -224,6 +226,9 @@ class BrowserManager:
 
             # Create page
             self._page = await self._context.new_page()
+            
+            # Update page wait manager with new page
+            self.page_wait.update_page(self._page)
 
             # 🔥 STEALTH ALWAYS APPLIED TO EVERY PAGE!
             stealth_success = await self.stealth_manager.apply_stealth(self._page)
@@ -290,7 +295,47 @@ class BrowserManager:
         return options
 
     async def navigate_async(self, url: str, wait_for: Optional[str] = None) -> Dict[str, Any]:
-        """Navigate to URL with basic error handling"""
+        """Navigate to URL with stealth retry logic"""
+        # Always use stealth retry logic
+        return await self._navigate_with_stealth_retry(url, wait_for)
+
+    async def _navigate_with_stealth_retry(self, url: str, wait_for: Optional[str] = None) -> Dict[str, Any]:
+        """Navigate with stealth retry logic - universal for all sites"""
+        self.logger_bridge.log_info(f"🥷 Stealth navigation to: {url}")
+        
+        # Stealth warmup if enabled
+        if self.config.stealth_warmup_enabled:
+            self.logger_bridge.log_info(f"🧪 First visiting stealth test page: {self.config.stealth_test_url}")
+            await self._navigate_basic(self.config.stealth_test_url, None)
+            self.logger_bridge.log_info(f"⏳ Waiting {self.config.stealth_warmup_delay} seconds for stealth establishment...")
+            await asyncio.sleep(self.config.stealth_warmup_delay)
+        
+        # Now navigate to the actual target URL with retry logic
+        max_retries = self.config.stealth_retry_attempts
+        for attempt in range(max_retries + 1):
+            if attempt > 0:
+                self.logger_bridge.log_info(f"🔄 Stealth retry attempt {attempt}/{max_retries}")
+                await asyncio.sleep(self.config.stealth_retry_delay)
+            
+            result = await self._navigate_basic(url, wait_for)
+            
+            # If successful, return result
+            if result["success"]:
+                if attempt > 0:
+                    self.logger_bridge.log_info(f"✅ Stealth retry successful on attempt {attempt + 1}")
+                return result
+            
+            # If failed and we have retries left, continue
+            if attempt < max_retries:
+                self.logger_bridge.log_info(f"⚠️ Navigation failed, will retry in {self.config.stealth_retry_delay} seconds...")
+                continue
+        
+        # All retries failed
+        self.logger_bridge.log_warning(f"❌ Stealth navigation failed after {max_retries + 1} attempts")
+        return result
+
+    async def _navigate_basic(self, url: str, wait_for: Optional[str] = None) -> Dict[str, Any]:
+        """Basic navigation without retry logic"""
         if not self._initialized or not self._page:
             raise RuntimeError("Browser not initialized. Call initialize_async() first.")
 
@@ -328,13 +373,13 @@ class BrowserManager:
             title = await self._page.title()
             self.logger_bridge.log_navigation_success(self._page.url, title, duration_ms)
 
-            # Check for captcha after navigation
-            captcha_result = await self.captcha_manager.detect_captcha(self._page)
-            if captcha_result.detected:
-                self.logger_bridge.log_captcha_detected(captcha_result)
-                self.logger_bridge.log_warning(f"⚠️ Captcha detected: {captcha_result.captcha_type.value}")
-                # Update session status to indicate captcha is required
-                self.session_metadata.current_status = BrowserSessionStatus.CAPTCHA_REQUIRED
+            # Check for captcha after navigation (TEMPORARILY DISABLED FOR DEBUGGING)
+            # captcha_result = await self.captcha_manager.detect_captcha(self._page)
+            # if captcha_result.detected:
+            #     self.logger_bridge.log_captcha_detected(captcha_result)
+            #     self.logger_bridge.log_warning(f"⚠️ Captcha detected: {captcha_result.captcha_type.value}")
+            #     # Update session status to indicate captcha is required
+            #     self.session_metadata.current_status = BrowserSessionStatus.CAPTCHA_REQUIRED
 
             return {
                 "success": True,
@@ -366,6 +411,10 @@ class BrowserManager:
                 "error": str(e),
             }
 
+    async def wait_for_page_ready_async(self, wait_type: str = "networkidle", timeout: int = 10000) -> bool:
+        """Wait for page to be ready for parsing (legacy method - use page_wait.* methods instead)"""
+        return await self.page_wait.wait_custom(wait_type, timeout)
+
     async def get_page_content_async(self) -> Optional[str]:
         """Get current page content"""
         if not self._page:
@@ -581,6 +630,7 @@ class BrowserManager:
                     self.logger_bridge.log_warning(f"⚠️ Page already closed: {e}")
                 finally:
                     self._page = None
+                    self.page_wait.update_page(None)
 
             # Close context with safety checks
             if self._context:

unrealon_browser/dto/models/config.py

@@ -31,3 +31,10 @@ class BrowserConfig(BaseModel):
     # Performance
     disable_images: bool = Field(default=False)
     enable_stealth_check: bool = Field(default=False)
+    
+    # Stealth settings
+    stealth_warmup_enabled: bool = Field(default=True, description="Enable stealth warmup before target navigation")
+    stealth_test_url: str = Field(default="https://bot.sannysoft.com", description="URL for stealth warmup")
+    stealth_warmup_delay: float = Field(default=3.0, description="Delay in seconds after stealth warmup")
+    stealth_retry_attempts: int = Field(default=2, description="Maximum retry attempts for failed navigation")
+    stealth_retry_delay: float = Field(default=3.0, description="Delay between retry attempts")

unrealon_browser/managers/__init__.py

@@ -7,6 +7,8 @@ from .profile import ProfileManager
 from .logger_bridge import BrowserLoggerBridge, create_browser_logger_bridge
 from .cookies import CookieManager
 from .captcha import CaptchaDetector
+from .page_wait_manager import PageWaitManager
+
 
 __all__ = [
     "StealthManager",
@@ -15,4 +17,5 @@ __all__ = [
     "create_browser_logger_bridge",
     "CookieManager",
     "CaptchaDetector",
+    "PageWaitManager",
 ]

unrealon_browser/managers/logger_bridge.py

@@ -141,7 +141,7 @@ class BrowserLoggerBridge:
             session_id=metadata.session_id,
             parser_name=metadata.parser_name,
             browser_type=metadata.browser_type or "unknown",
-            stealth_level="unknown",
+            stealth_level="ALWAYS_ON",  # 🔥 STEALTH ALWAYS ON!
             proxy_host=getattr(metadata.proxy, "host", None) if metadata.proxy else None,
             proxy_port=getattr(metadata.proxy, "port", None) if metadata.proxy else None,
         )
@@ -168,20 +168,20 @@ class BrowserLoggerBridge:
             navigation_type="browser_navigation",
         )
 
-    def log_stealth_applied(self, stealth_level: str, success: bool) -> None:
+    def log_stealth_applied(self, stealth_level: str = "ALWAYS_ON", success: bool = True) -> None:
         """Log stealth application - 🔥 STEALTH ALWAYS ON!"""
         self._browser_events["stealth_applied"] += 1
 
         if success:
             self._log_info(
-                f"Stealth measures applied: {stealth_level}",
-                stealth_level=stealth_level,
+                "Stealth measures applied: ALWAYS_ON",
+                stealth_level="ALWAYS_ON",
                 stealth_success=True,
             )
         else:
             self._log_warning(
-                f"Stealth application failed: {stealth_level}",
-                stealth_level=stealth_level,
+                "Stealth application failed: ALWAYS_ON",
+                stealth_level="ALWAYS_ON",
                 stealth_success=False,
             )
 

unrealon_browser/managers/page_wait_manager.py

@@ -0,0 +1,198 @@
+"""
+Page Wait Manager - Convenient methods for different page loading scenarios
+"""
+import asyncio
+from typing import Optional
+from playwright.async_api import Page
+
+from .logger_bridge import BrowserLoggerBridge as LoggingBridge
+
+
+class PageWaitManager:
+    """Manager for different page waiting strategies"""
+    
+    def __init__(self, page: Optional[Page], logger_bridge: LoggingBridge):
+        self._page = page
+        self.logger_bridge = logger_bridge
+    
+    def update_page(self, page: Optional[Page]):
+        """Update the page reference"""
+        self._page = page
+    
+    # Quick wait methods with fallback
+    async def wait_fast_with_fallback(self) -> bool:
+        """Fast wait - networkidle 5s, fallback to domcontentloaded 3s"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info("🚀 Fast wait with fallback (networkidle 5s → domcontentloaded 3s)")
+        
+        # Try networkidle first
+        if await self._wait_for_state("networkidle", 5000):
+            return True
+            
+        # Fallback to domcontentloaded
+        self.logger_bridge.log_info("⏳ Networkidle timeout, trying domcontentloaded...")
+        return await self._wait_for_state("domcontentloaded", 3000)
+    
+    async def wait_safe_with_fallback(self) -> bool:
+        """Safe wait - networkidle 10s, fallback to domcontentloaded 5s"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info("🛡️ Safe wait with fallback (networkidle 10s → domcontentloaded 5s)")
+        
+        # Try networkidle first
+        if await self._wait_for_state("networkidle", 10000):
+            return True
+            
+        # Fallback to domcontentloaded
+        self.logger_bridge.log_info("⏳ Networkidle timeout, trying domcontentloaded...")
+        return await self._wait_for_state("domcontentloaded", 5000)
+    
+    # Generic methods
+    async def wait_fast(self) -> bool:
+        """Fast wait - domcontentloaded 3s"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info("⚡ Fast wait (domcontentloaded 3s)")
+        return await self._wait_for_state("domcontentloaded", 3000)
+    
+    async def wait_standard(self) -> bool:
+        """Standard wait - networkidle 10s"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info("⏳ Standard wait (networkidle 10s)")
+        return await self._wait_for_state("networkidle", 10000)
+    
+    async def wait_full_load(self) -> bool:
+        """Full page load including images - load 30s"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info("🖼️ Full load wait (load 30s)")
+        return await self._wait_for_state("load", 30000)
+    
+    async def wait_minimal(self) -> bool:
+        """Minimal wait - domcontentloaded 1s"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info("⚡ Minimal wait (domcontentloaded 1s)")
+        return await self._wait_for_state("domcontentloaded", 1000)
+    
+    # Specialized methods
+    async def wait_spa(self) -> bool:
+        """Single Page Application wait - networkidle 15s"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info("⚛️ SPA wait (networkidle 15s)")
+        return await self._wait_for_state("networkidle", 15000)
+    
+    # Custom methods
+    async def wait_custom(self, wait_type: str = "networkidle", timeout: int = 10000) -> bool:
+        """Custom wait with specified parameters"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info(f"🔧 Custom wait ({wait_type} {timeout}ms)")
+        return await self._wait_for_state(wait_type, timeout)
+    
+    async def wait_with_fallback(self, 
+                                primary_type: str = "networkidle", 
+                                primary_timeout: int = 10000,
+                                fallback_type: str = "domcontentloaded", 
+                                fallback_timeout: int = 5000) -> bool:
+        """Wait with custom fallback strategy"""
+        if not self._page:
+            return False
+            
+        self.logger_bridge.log_info(f"🔄 Fallback wait ({primary_type} {primary_timeout}ms → {fallback_type} {fallback_timeout}ms)")
+        
+        # Try primary first
+        if await self._wait_for_state(primary_type, primary_timeout):
+            return True
+            
+        # Fallback
+        self.logger_bridge.log_info(f"⏳ {primary_type} timeout, trying {fallback_type}...")
+        return await self._wait_for_state(fallback_type, fallback_timeout)
+    
+    # Helper methods
+    async def _wait_for_state(self, wait_type: str, timeout: int) -> bool:
+        """Internal method to wait for specific state"""
+        if not self._page:
+            return False
+            
+        try:
+            if wait_type == "networkidle":
+                await self._page.wait_for_load_state("networkidle", timeout=timeout)
+            elif wait_type == "domcontentloaded":
+                await self._page.wait_for_load_state("domcontentloaded", timeout=timeout)
+            elif wait_type == "load":
+                await self._page.wait_for_load_state("load", timeout=timeout)
+            else:
+                # Default to networkidle
+                await self._page.wait_for_load_state("networkidle", timeout=timeout)
+            
+            self.logger_bridge.log_info(f"✅ Page ready ({wait_type})")
+            return True
+            
+        except Exception as e:
+            self.logger_bridge.log_warning(f"⚠️ Page ready timeout ({wait_type}): {e}")
+            return False
+    
+    # Utility methods
+    async def wait_for_selector(self, selector: str, timeout: int = 10000) -> bool:
+        """Wait for specific selector to appear"""
+        if not self._page:
+            return False
+            
+        try:
+            self.logger_bridge.log_info(f"🎯 Waiting for selector: {selector}")
+            await self._page.wait_for_selector(selector, timeout=timeout)
+            self.logger_bridge.log_info(f"✅ Selector found: {selector}")
+            return True
+        except Exception as e:
+            self.logger_bridge.log_warning(f"⚠️ Selector timeout: {selector} - {e}")
+            return False
+    
+    async def wait_for_text(self, text: str, timeout: int = 10000) -> bool:
+        """Wait for specific text to appear on page"""
+        if not self._page:
+            return False
+            
+        try:
+            self.logger_bridge.log_info(f"📝 Waiting for text: {text}")
+            await self._page.wait_for_function(
+                f"document.body.innerText.includes('{text}')",
+                timeout=timeout
+            )
+            self.logger_bridge.log_info(f"✅ Text found: {text}")
+            return True
+        except Exception as e:
+            self.logger_bridge.log_warning(f"⚠️ Text timeout: {text} - {e}")
+            return False
+    
+    async def wait_for_url_change(self, timeout: int = 10000) -> bool:
+        """Wait for URL to change"""
+        if not self._page:
+            return False
+            
+        try:
+            current_url = self._page.url
+            self.logger_bridge.log_info(f"🔗 Waiting for URL change from: {current_url}")
+            
+            await self._page.wait_for_function(
+                f"window.location.href !== '{current_url}'",
+                timeout=timeout
+            )
+            
+            new_url = self._page.url
+            self.logger_bridge.log_info(f"✅ URL changed to: {new_url}")
+            return True
+        except Exception as e:
+            self.logger_bridge.log_warning(f"⚠️ URL change timeout: {e}")
+            return False

unrealon_driver/__init__.py

@@ -14,9 +14,12 @@ Key Features:
 - 🛡️ Type Safety: Full Pydantic v2 compliance
 """
 
-from unrealon import VersionInfo
+try:
+    from importlib.metadata import version
 
-__version__ = VersionInfo().version
+    __version__ = version("unrealon-driver")
+except Exception:
+    __version__ = "1.0.0-dev"
 
 
 from .parser import (
@@ -33,8 +36,6 @@ from .parser import (
     ErrorManager,
     RetryConfig,
     ErrorInfo,
-
-
 )
 from .exceptions import ParserError, BrowserError
 
@@ -55,8 +56,6 @@ __all__ = [
     "ErrorManager",
     "RetryConfig",
     "ErrorInfo",
-
-
     # Exceptions
     "ParserError",
     "BrowserError",

unrealon/__init__.py

@@ -1,40 +0,0 @@
-"""
-UnrealOn - Main Package
-Enterprise browser automation framework with WebSocket bridge for distributed web scraping.
-"""
-
-from importlib.metadata import version
-from pydantic import BaseModel, Field, ConfigDict
-
-
-try:
-    __version__ = version("unrealon")
-except Exception:
-    __version__ = "1.1.4"
-
-
-class VersionInfo(BaseModel):
-    """Version information model."""
-
-    model_config = ConfigDict(validate_assignment=True, extra="forbid")
-
-    version: str = Field(default=__version__)
-
-
-# Re-export main modules
-import unrealon_driver
-import unrealon_server
-import unrealon_browser
-
-# Re-export all from submodules
-from unrealon_driver import *
-from unrealon_server import *
-from unrealon_browser import *
-
-__all__ = [
-    "VersionInfo",
-    # Re-export all from submodules
-    *getattr(unrealon_driver, "__all__", []),
-    *getattr(unrealon_server, "__all__", []),
-    *getattr(unrealon_browser, "__all__", []),
-]