agentic-stealth-browser 2.3.0__tar.gz → 2.5.0__tar.gz

{agentic_stealth_browser-2.3.0 → agentic_stealth_browser-2.5.0}/CHANGELOG.md

@@ -7,6 +7,113 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ---
 
+## [Unreleased]
+
+## [2.5.0] — Real Dashboard Backend Adapters (#444) (2026-06-07)
+
+### Added
+- **`BackendAdapter` Protocol + `Capability` enum** (`production/adapters/base.py:84`): runtime-checkable protocol with `launch`/`close`/`navigate`/`click`/`fill`/`screenshot`/`status`/`capabilities` surface. All three backends (M1/M2/M3) implement it. New `AdapterLaunchError` and `AdapterCapabilityError` exception types; `register_adapter` enforces non-empty string names. (#444 M0)
+- **`CDPBridgeAdapter`** (`production/adapters/cdp_bridge.py`): Playwright `connect_over_cdp` over WebSocket. Used for adopted-context / WSL-host workflows where the operator already owns a browser process. (#444 M1)
+- **`PlaywrightMCPAdapter`** (`production/adapters/playwright_mcp.py`): stdio subprocess + MCP JSON-RPC. Spawns `npx @playwright/mcp` with an explicit env allowlist (PATH, HOME) and a 16 MB `MAX_FRAME_BYTES` frame cap. (#444 M2)
+- **`AgenticStealthMCPAdapter`** (`production/adapters/agentic_stealth_mcp.py`): stdio subprocess + MCP JSON-RPC, round-trips through the bundled `production.mcp_server` runtime. Same env allowlist, frame cap, and stderr-drain guarantees as M2. (#444 M3)
+- **Dashboard backend wiring (M4)**: `BrowserRuntimeManager` resolves the configured backend through the adapter registry, records an audit event on backend switch, surfaces negotiated capabilities in dashboard status, and rejects unsupported actions with a backend-specific error. `production/dashboard_adapter_bridge.py` is now a thin shim that re-exports the three M1–M3 adapters. (#444 M4)
+
+### Changed
+- `production/hermes_dashboard.py` re-exports `BackendAdapter` from `production/adapters/`.
+- `production/dashboard_adapter_bridge.py` thin shim replaces the three alias backends.
+- `MANIFEST.in`: ships `assets/*.{gif,png,jpg,svg}` (demo GIF), `docs/`, `CODE_OF_CONDUCT.md`, `CONTRIBUTING.md`, `SECURITY.md` (sdist previously dropped the demo GIF and the entire `docs/` tree).
+- `README.md`: added "What is this / When to use / When NOT to use" block at top, honest comparison table (region client-hello profiles, not full uTLS), install-from-source section, full project tree + doc index. Dropped unverified test-count badge and Buy Me A Coffee. 177 → 238 lines.
+- New `docs/README.md` operator/reference/planning index. Root-level `*PLAN.md` / `*READINESS.md` / `*DASHBOARD.md` / `HN_POST.md` moved to `docs/plans/misc/` as historical.
+- Examples rewritten for honesty and reproducibility: `01_cloudflare_bypass.py` (nowsecure.nl + screenshot), `02_linkedin_search.py` (`linkedin_2026` preset + webdriver check + DDG), `03_amazon_product.py` (product → title + price). `recipes/README.md` table updated.
+- `pyproject.toml` version 2.4.1 → 2.5.0.
+
+### Fixed
+- **#444 M0 review (showstoppers)**:
+  - **S1 capability gating**: M2/M3 `navigate`/`click`/`fill`/`screenshot` now call `self.supports()` and raise `AdapterCapabilityError` if the adapter doesn't declare the capability (enforces `base.py:84` protocol contract). New `_require_capability()` helper on each adapter.
+  - **S2 stderr drain**: M2/M3 `stderr=PIPE` is now drained by a background task started right after `spawn` and cancelled in `close()`. A noisy child no longer fills the ~64 KB kernel pipe buffer, blocks on its next stderr write, hangs the handshake, and triggers a slow SIGKILL on close.
+  - **F1 stale state on retry**: after MCP handshake failure, `terminate_subprocess()` is called and `self._proc` / `self._client` are cleared. A caller that catches `AdapterLaunchError` and then calls `close()` no longer operates on a dead `Process` handle.
+- **#444 M0 review (smaller)**:
+  - **F2 frame cap**: `_jsonrpc_stdio.readline()` now enforces `MAX_FRAME_BYTES = 16 MB` and raises `ConnectionError` on overflow. A misbehaving server emitting a multi-GB line no longer OOMs the client.
+  - **F3 env allowlist**: both stdio adapters now pass an explicit `env={PATH, HOME}` to `asyncio.create_subprocess_exec` instead of inheriting the operator's full environment (`OPENAI_API_KEY`, `AWS_*`, `DATABASE_URL`, ...).
+  - **A3 dead tautology**: `cdp_bridge.screenshot()` had a defensive `Capability.SCREENSHOT` check that could never fire (always in `capabilities()`). Branch removed; M0 contract is the only contract that matters.
+  - **M0 code-quality review (17abc50)**: exception types narrowed to library-specific where appropriate, `isinstance` coverage tightened, defensive tautology branches removed, transport teardown made symmetric with launch.
+- **Adapter capability gating**: now consistent across M1/M2/M3 — each action is contractually declared, not assumed.
+- **Ruff/format housekeeping** (4432701): 12 ruff format issues + 1 unused import fixed.
+
+### Tests / CI
+- `tests/test_backend_adapter_contract.py` — new contract test pinning the M0 Protocol surface, `Capability` enum members, and `register_adapter` duplicate/empty/non-string rejection.
+- `tests/test_adapter_cdp_bridge.py`, `tests/test_adapter_playwright_mcp.py`, `tests/test_adapter_agentic_stealth_mcp.py` — new per-adapter tests covering launch, capability negotiation, action gating, stderr drain, env allowlist, and close lifecycle.
+- `tests/test_dashboard_protocol_bridge.py` — new dashboard-wiring test (M4): audit event on backend switch, capability surface, unsupported-action rejection.
+- `pytest -q`: 1025 collected, 1023 passed, 2 e2e skipped, 0 failed (139 s; baseline maintained).
+- `ruff check` + `ruff format --check` clean across 159 Python files.
+
+Closes #444.
+
+---
+
+## [2.4.1] — v2.4.0 Stabilization Patch (2026-06-03)
+
+### Fixed / Security / Reliability (P0/P1 per #459)
+- **#454** MCP nav/scrape: `is_url_safe` now fails closed for non-http(s) schemes (`file:`, `javascript:`, `data:`, `ftp:`, ...), missing hosts, DNS errors, and empty resolutions. Unsafe inputs raise `MCP_SSRF_BLOCKED` before any browser navigation. Added unit matrix + dispatch regression tests.
+- **#455** MCP runtime: declarative input validation from `production/mcp_input_validator.py` is now enforced in `tools/call` dispatch (types, lengths, ranges, required, patterns, `additionalProperties: false`). Validation errors return stable `MCP_VALIDATION_ERROR` without invoking handlers. Schema/validator parity and dispatch tests added.
+- **#451** CDP attach: adopted user tabs (`new_context=False`) are no longer closed on `close()`. New `_owns_page` tracking + conditional teardown; adopted contexts/pages preserved per contract. Updated tests + docs.
+- **#452** CDP attach: post-`connect_over_cdp` failures (bad context_index, new_context, page create, stealth) now best-effort rollback `_pw`, `_remote_browser`, owned ctxs (never adopted), and reset attach flags for clean retry.
+- **#453** CDP attach: attach path now initializes `human`, `orchestrator`, `logger`, `scraper`, `ai`, `recovery` (with attach degradation notes). `safe_goto`/`safe_click`/`safe_type` + MCP `stealth_scrape` now operational on attached sessions.
+- **#458** Hermes dashboard: startup rejects default/empty/`change-me` password when binding to non-loopback (0.0.0.0, RFC1918, public, IPv6 non-loop). Loopback dev still convenient. Added guard tests.
+- **#457** CLI `scrape`: removed double `launch()` (async-with already launches; now passes preset/region via ctor so aenter applies them). Exactly one launch + one close. Aligned all user-facing names/help/examples to `agentic-stealth-browser`. Mocked regression test.
+- **#456** Packaging: `pyproject.toml` now includes `production.*`, `workflows.*`, `plugins.*`. `MANIFEST.in` extended. CI verify expanded for SDK import + bundled workflow discovery from clean cwd. MCP server workflow root switched to user-writable `~/.agentic-browser/...` + bundled discovery fallback for install safety.
+- **#447** MCP version: `SERVER_VERSION` now derives from `importlib.metadata.version("agentic-stealth-browser")` (falls back to 2.4.1); no longer stale.
+- **#448** CDP attach gate: relaxed RFC1918 rejection under `allow_remote=true` for attach (WSL host IPs are intentionally private); link-local/cloud still blocked. Nav remains strict. Reconciled docs + tests with real WSL workflow.
+- **#450** Docs/metadata: toned down TLS/JA3/JA4 claims to "region client-hello profiles (process-level)"; attach degradation already documented the limits. No wire-level full-stack spoofing claimed.
+- **#449** Release hygiene: refreshed counts, changelog, version to 2.4.1, support notes, publish smoke.
+
+### Added
+- Runtime helpers and ownership tracking for attach mode.
+- Validation dispatch + matrix tests; single-launch CLI mock test; dashboard bind security tests.
+
+### Changed
+- `pyproject.toml` version 2.4.0 → 2.4.1; packages.find + MANIFEST.in for full artifacts.
+- Attach lifecycle docs/examples (no more auto-launch + attach in same `async with`).
+- MCP attach remote safety contract + tests for WSL compatibility.
+
+### Tests / CI
+- All non-e2e/non-live green.
+- Focused attach + MCP safety + validation + CLI mocks + dashboard guard.
+- `ruff check` + `ruff format --check` pass.
+- Wheel build + import + discovery smoke updated in publish.yml.
+
+Closes #447, #448, #449, #450, #451, #452, #453, #454, #455, #456, #457, #458, #459 (patch scope).
+
+---
+
+## [2.4.0] — Attach-Mode Hardening (2026-06-01)
+
+### Fixed
+- **#438** `stealth_attach_over_cdp` loopback gate reused `is_url_safe` — DNS rebinding / hostname→private-IP bypass closed. Replaces string match with full RFC-1918 + link-local + cloud-metadata check. The gate is now a two-layer check: `is_loopback_host` first, then `is_url_safe` only when `allow_remote=true` (so even explicitly-allowed remote hosts can't be RFC-1918 or link-local).
+- **#441** Link-local IPv6 (`fe80::/10`) added to `_BLOCKED_NETWORKS`. Side-effect of #438's refactor — `is_url_safe` now rejects IPv6 link-local the same way it rejects IPv4 `169.254.0.0/16`.
+- **#440** `add_init_script` install failure now surfaces in the return payload as `stealth_applied: false` + `stealth_error: "<ExceptionType>: <message>"`. Previously silent. New `stealth_requested` field distinguishes caller intent from actual install result.
+- **#439** `AgentBrowser.close()` teardown logic now uses a typed `TeardownMode` enum (`LAUNCHED | POOLED | ATTACHED_OWNED_CTX | ATTACHED_ADOPTED_CTX`) instead of 3 scattered `getattr` flag checks. Each branch owns exactly one teardown. Easier to reason about, easier to extend.
+
+### Added
+- **`is_loopback_host(url: str) -> bool`** helper in `production/mcp_server.py` — literal/IP/DNS loopback check. Used by the attach loopback gate.
+- **6 new tests in `tests/test_mcp_url_safety.py`** covering loopback literal, `localhost`, bare `host:port` normalization, RFC-1918 rejection, link-local IPv6 rejection.
+- **4 new tests in `tests/test_attach_over_cdp.py`** covering the two-layer gate (RFC-1918, link-local IPv6, with/without `allow_remote=true`).
+- **4 new tests in `tests/test_attach_over_cdp.py`** for `TeardownMode` enum and stealth-failure surface (monkeypatched `add_init_script` rejection).
+
+### Changed
+- `production/mcp_server.py`: new `is_loopback_host` helper. `_BLOCKED_NETWORKS` extended with `fe80::/10`. `_tool_stealth_attach_over_cdp` rewritten as a two-layer gate.
+- `core/agent_browser.py`: new `TeardownMode` enum. `close()` branches on the enum. `attach_over_cdp` return dict now includes `stealth_requested` and `stealth_error` fields.
+- `pyproject.toml`: `version` bumped 2.3.0 → 2.4.0.
+
+### Test coverage
+- 22/22 pass in `tests/test_attach_over_cdp.py` + `tests/test_mcp_url_safety.py`
+- No new regressions in the unit-test suite (3 pre-existing failures on master — `test_human_behavior_fuzz` MockPage signature mismatch + `test_phase7_fixes` merge conflict — both unrelated to this release)
+
+### Closes
+- #438, #439, #440, #441
+
+---
+
 ## [2.3.0] — Show HN & Community Launch (2026-05-28)
 
 ### Added

agentic_stealth_browser-2.5.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,70 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[GitHub Issues](https://github.com/shanewas/agentic-stealth-browser/issues).
+All complaints will be reviewed and investigated promptly and fairly.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html

agentic_stealth_browser-2.5.0/CONTRIBUTING.md

@@ -0,0 +1,237 @@
+# CONTRIBUTING.md (#55)
+
+Thank you for your interest in contributing to the Agentic Stealth Browser! This guide will help you get started.
+
+---
+
+## Development Setup
+
+### 1. Clone the Repository
+
+```bash
+git clone https://github.com/shanewas/agentic-stealth-browser.git
+cd agentic-stealth-browser
+```
+
+### 2. Create a Virtual Environment
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+```
+
+### 3. Install Dependencies
+
+```bash
+pip install -e ".[dev]"
+playwright install chromium
+```
+
+### 4. Run Tests
+
+```bash
+# Run all tests
+pytest tests/ -v
+
+# Run specific test files
+pytest tests/test_stealth_modules.py -v
+pytest tests/test_contract_agent_browser.py -v
+pytest tests/test_recovery_phase1.py -v
+
+# Run with coverage
+pytest tests/ --cov=core --cov=stealth --cov=recovery --cov=behavior
+```
+
+---
+
+## Code Style
+
+### Formatting
+
+We use standard Python conventions:
+- 4-space indentation
+- Maximum line length: 120 characters
+- Type hints for all public APIs
+- Docstrings for all public methods
+
+### Naming Conventions
+
+- Classes: `PascalCase` (e.g., `AgentBrowser`, `StealthConfig`)
+- Functions/Methods: `snake_case` (e.g., `safe_goto`, `warm_up_before_work`)
+- Constants: `UPPER_SNAKE_CASE` (e.g., `DEFAULT_PERSONA`, `MAX_RETRIES`)
+- Private methods: Leading underscore (e.g., `_run_step`, `_check_circuit_breaker`)
+
+---
+
+## Making Changes
+
+### 1. Create a Branch
+
+```bash
+git checkout -b feat/your-feature-name
+# or
+git checkout -b fix/issue-description
+```
+
+### 2. Make Your Changes
+
+- Follow the existing code style
+- Add tests for new functionality
+- Update documentation if needed
+
+### 3. Run Tests
+
+```bash
+# Ensure all tests pass
+pytest tests/ -v
+
+# Check for any import issues
+python -c "from core.agent_browser import AgentBrowser"
+python -c "from stealth.advanced_stealth import get_stealth_script"
+```
+
+### 4. Commit Your Changes
+
+```bash
+git add .
+git commit -m "feat: description of your change
+
+More detailed description if needed.
+
+Closes #123"
+```
+
+### 5. Push and Create a Pull Request
+
+```bash
+git push origin your-branch-name
+```
+
+Then open a PR on GitHub.
+
+---
+
+## Pull Request Guidelines
+
+### Title Format
+
+- `feat: ` for new features
+- `fix: ` for bug fixes
+- `docs: ` for documentation changes
+- `test: ` for test additions/changes
+- `refactor: ` for code refactoring
+- `perf: ` for performance improvements
+- `security: ` for security fixes
+
+### Description
+
+Include:
+- What this PR does
+- Why this change is needed
+- Any related issues (e.g., "Closes #123")
+- Testing instructions
+
+### Requirements
+
+- [ ] All tests pass
+- [ ] New code has tests
+- [ ] Documentation is updated (if applicable)
+- [ ] No sensitive data in code or commits
+
+---
+
+## Architecture Overview
+
+```
+agentic-stealth-browser/
+├── core/
+│   ├── agent_browser.py      # Main AgentBrowser class
+│   ├── error_messages.py     # User-friendly error messages
+│   └── types.py              # Type hints and stubs
+├── stealth/
+│   ├── advanced_stealth.py   # Stealth script generation
+│   ├── tls_fingerprint.py    # TLS profile management
+│   ├── headers.py            # HTTP header spoofing
+│   ├── profiles.py           # Persona/DeviceProfile system
+│   └── presets.py            # Platform presets
+├── behavior/
+│   ├── human_behavior.py     # Human-like behavior simulation
+│   └── orchestration.py      # Behavior orchestration
+├── recovery/
+│   ├── anti_block_orchestrator.py  # Block detection & recovery
+│   └── explain_blocked.py          # Block explanation
+├── proxy/
+│   └── proxy_manager.py      # Proxy management & rotation
+├── sessions/
+│   ├── session_manager.py    # Session management
+│   └── cookie_manager.py     # Cookie management
+├── production/
+│   ├── metrics.py            # Metrics collection
+│   ├── rate_limiter.py       # Rate limiting
+│   ├── cli.py                # CLI entry point
+│   └── agent_orchestrator.py # Multi-agent orchestration
+├── audit/
+│   └── logger.py             # Audit logging
+├── scraping/
+│   └── scraper.py            # Stealth scraping
+├── ai/
+│   └── ai_hooks.py           # AI integration hooks
+├── tests/                    # Test suite
+# (documentation lives in README.md and git history)
+└── mcp_security.py           # MCP security hardening
+```
+
+---
+
+## Adding New Features
+
+### New Stealth Patches
+
+1. Add the patch to `stealth/advanced_stealth.py`
+2. Add tests to `tests/test_stealth_modules.py`
+3. Update relevant sections in README.md if behavior changes
+
+### New Platform Preset
+
+1. Add the preset to `stealth/presets.py`
+2. Add a recipe to `README.md`
+3. Test against the target site
+
+### New Recovery Strategy
+
+1. Update `PLATFORM_STRATEGIES` in `recovery/anti_block_orchestrator.py`
+2. Add tests to `tests/test_recovery_phase1.py`
+3. Document rate limiting behavior changes in README.md or relevant code comments
+
+### New Human Behavior
+
+1. Add the behavior to `behavior/human_behavior.py`
+2. Respect `self.realism_level` for CI/light mode
+3. Add tests
+
+---
+
+## Reporting Issues
+
+When reporting a bug, include:
+- Python version
+- Playwright version
+- Error message and stack trace
+- Code snippet that reproduces the issue
+- Debug report (`await browser.debug_report(print_report=True)`)
+- Health status (`await browser.get_health_status()`)
+
+---
+
+## Security
+
+If you discover a security vulnerability:
+1. **Do not** open a public issue
+2. Email the maintainer directly
+3. Include steps to reproduce and potential impact
+
+---
+
+## License
+
+By contributing, you agree that your contributions will be licensed under the MIT License.

agentic_stealth_browser-2.5.0/MANIFEST.in

@@ -0,0 +1,19 @@
+include LICENSE
+include README.md
+include CHANGELOG.md
+include CODE_OF_CONDUCT.md
+include CONTRIBUTING.md
+include SECURITY.md
+include .env.example
+
+include assets/*.gif assets/*.png assets/*.jpg assets/*.svg
+
+recursive-include docs *.md *.html *.svg *.jsonl
+recursive-include stealth *.js *.json
+recursive-include production Dockerfile docker-healthcheck.py
+recursive-include examples *.md *.py *.txt
+recursive-include tests *.py *.json
+recursive-include workflows *.yaml *.yml
+recursive-include plugins *.py
+recursive-include production/sdk *.py
+recursive-include scripts *.sh *.py