oh-my-opencode 4.12.0 → 4.13.0

package/dist/skills/ultimate-browsing/ATTRIBUTION.md

@@ -0,0 +1,105 @@
+# ATTRIBUTION / NOTICE
+
+This skill (`ultimate-browsing`, part of `@oh-my-opencode/shared-skills`) ships
+project-original content plus two third-party tools that it installs at runtime
+(it does NOT vendor their source). Each runtime tool's license and required
+notices are reproduced below.
+
+---
+
+## 1. Project-original content (no third-party source vendored)
+
+The following are authored by the oh-my-openagent project and carry no third-party
+license obligation:
+
+- `engine/**` — the insane-search Tier-1 fetch engine (curl_cffi grid, WAF
+  detection, Playwright fallback templates, `bias_check.py` no-site-name gate).
+- `references/insane-search/**` and `references/agent-reach/**` — the Tier-1 and
+  Tier-1.5 reference docs.
+- `scripts/extract_cookies.py`, `scripts/cookie_paths.py`, `scripts/cookie_crypto.py`
+  and their tests — the cross-platform cookie module.
+- `SKILL.md`, `references/chrome-stealth.md`.
+
+These reference platform-native CLIs and public APIs by name (e.g. `xhs`, `yt-dlp`,
+`agent-reach`, `mcporter`, Jina Reader, V2EX public API). Those are external tools the
+user installs separately; this skill includes none of their source.
+
+---
+
+## 2. CloakBrowser (CloakHQ) — Tier-2 stealth Chromium (runtime dependency)
+
+The Tier-2 stealth browser is **CloakBrowser**, installed at runtime via `pip`
+(`pip install cloakbrowser`). No CloakBrowser source is vendored in this repository.
+
+- Source: https://github.com/CloakHQ/CloakBrowser
+- Pinned runtime version: **0.4.0** (documented in `references/chrome-stealth.md`;
+  this is a documented version string, not an automated drift check).
+- Wrapper source license: MIT License.
+- Binary license: the compiled CloakBrowser Chromium binary downloaded by
+  `cloakbrowser.ensure_binary()` is governed by the separate CloakBrowser
+  Binary License:
+  https://github.com/CloakHQ/CloakBrowser/blob/main/BINARY-LICENSE.md
+- Redistribution note: this npm package does not redistribute the CloakBrowser
+  binary, does not repackage it, and does not include it in `skills/` or
+  `dist/skills`. Users who run the Tier-2 setup download the binary directly
+  from CloakHQ's official distribution channels and must comply with that
+  binary license.
+
+MIT wrapper source license:
+
+```
+MIT License
+
+Copyright (c) 2026 CloakHQ
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+```
+
+---
+
+## 3. agent-browser (vercel-labs) — Tier-2 CDP automation CLI (runtime dependency)
+
+The Tier-2 automation CLI is **agent-browser**, installed at runtime via `npm`
+(`npm i -g agent-browser`). No agent-browser source is vendored in this repository.
+
+- Source: https://github.com/vercel-labs/agent-browser
+- Pinned runtime version: **0.29.1** (documented in `references/chrome-stealth.md`;
+  documented version string, no automated drift check).
+- Licensed under the Apache License, Version 2.0 (the "License"); you may not use
+  these files except in compliance with the License. You may obtain a copy of the
+  License at:
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software distributed under
+  the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied. See the License for the specific language governing
+  permissions and limitations under the License.
+
+- **Changes (Apache-2.0 §4(b)):** none. agent-browser is installed unmodified at
+  runtime; no agent-browser source file is copied, modified, or redistributed by this
+  skill. This skill only documents how to invoke the upstream CLI.
+
+- **NOTICE:** the upstream agent-browser distribution may include a `NOTICE` file. As
+  this skill redistributes no agent-browser source, no upstream NOTICE content is
+  bundled here; consult the upstream repository for its `NOTICE` file when present.
+
+- **Trademark notice:** "agent-browser" and "Vercel" are referenced by name for
+  identification only. No trademark license is granted under the Apache License 2.0
+  (Section 6).

package/dist/skills/ultimate-browsing/SKILL.md

@@ -0,0 +1,140 @@
+---
+name: ultimate-browsing
+description: "Escalation skill for blocked or hard-to-reach web access — load it when a normal browse/fetch is blocked (WAF, 403, Cloudflare, JS-only render, login-gated, or a platform a generic fetcher cannot read). Tiered router: TIER 1 insane-search (headless extraction + WAF bypass via curl_cffi TLS impersonation, yt-dlp, Jina Reader, public APIs, Playwright real-Chrome fallback); TIER 1.5 agent-reach (platform-native readers for Chinese and social platforms: Xiaohongshu, Douyin, Weibo, Bilibili, V2EX, WeChat, plus Twitter/Reddit/LinkedIn/GitHub); TIER 2 Chrome stealth (CloakBrowser stealth Chromium + agent-browser CDP for clicks, forms, screenshots, video, cookie login). Triggers: blocked site, bypass bot detection, cloudflare/WAF bypass, scrape, stealth browser, import cookies, fill form, screenshot, play youtube, xiaohongshu, douyin, weibo, bilibili, v2ex, wechat article, podcast transcript. NOT for simple searches (use web-search) or plain fetches (use webfetch)."
+---
+
+# Ultimate Browsing
+
+Escalation web access for tasks a normal browse or fetch cannot complete. Reach for this skill the moment a page is blocked (WAF / 403 / Cloudflare), needs JS rendering, hides behind a login, or lives on a platform a generic fetcher cannot read. Escalate only when the cheaper tier cannot do the job:
+
+**Tier 1 — insane-search** (headless extraction + WAF bypass) -> **Tier 1.5 — agent-reach** (platform-native APIs, esp. Chinese platforms) -> **Tier 2 — Chrome stealth** (real interaction via CloakBrowser + agent-browser).
+
+## PHASE 0 — ROUTE FIRST (MANDATORY)
+
+```
+User request
+  |
+  +- extract text/data from a URL --------------------- TIER 1  insane-search
+  +- URL blocked / 403 / Cloudflare / WAF ------------- TIER 1  insane-search
+  +- YouTube/Vimeo/TikTok subtitles or metadata ------- TIER 1  insane-search (yt-dlp)
+  +- read an article / blog / Reddit / HN / arXiv ----- TIER 1  insane-search
+  |
+  +- Chinese platform (xhs/douyin/weibo/bilibili/v2ex/wechat)  TIER 1.5 agent-reach
+  +- podcast transcript / stock forum ----------------- TIER 1.5 agent-reach
+  +- Twitter feed / LinkedIn profile / GitHub via CLI - TIER 1.5 agent-reach
+  |
+  +- Tier 1/1.5 returned empty or partial ------------- TIER 2  Chrome stealth
+  +- click / fill form / scroll / interact ------------ TIER 2  Chrome stealth
+  +- screenshot / render / play video ----------------- TIER 2  Chrome stealth
+  +- login session across pages / inject cookies ------ TIER 2  Chrome stealth
+  +- test web app / QA / dogfood ---------------------- TIER 2  Chrome stealth
+  |
+  +- simple search query ------------------------------ NOT this skill (use web-search)
+```
+
+Read the matching reference before acting: [`references/insane-search/README.md`](references/insane-search/README.md), [`references/agent-reach/README.md`](references/agent-reach/README.md), or [`references/chrome-stealth.md`](references/chrome-stealth.md).
+
+## Tier 1 — insane-search (headless extraction)
+
+**When**: content extraction, blocked-URL bypass, media metadata — no browser UI needed.
+**Why first**: ~10x faster than a browser, no process spin-up; handles most "fetch this blocked page" requests via curl_cffi TLS impersonation, yt-dlp (1858 sites), Jina Reader, official public APIs, mobile URL transforms, and a Playwright real-Chrome fallback. The engine lives **inside this skill** at `engine/` and is invoked as a module.
+
+```bash
+# Core command — auto-detects WAF, runs the full fetch grid (run from the skill dir):
+python3 -m engine "https://example.com/blocked-page"
+#   add --selector "<CSS>" for positive-proof validation, --device auto|desktop|mobile,
+#   --trace to inspect every attempt, --json for machine-readable output.
+
+# YouTube subtitles / metadata (no browser):
+yt-dlp --write-sub --write-auto-sub --sub-lang "en,ko" --skip-download -o "/tmp/%(id)s" "<URL>"
+
+# Reddit / HN / Bluesky / arXiv etc. use official public endpoints — see the Phase 0 index in
+# references/insane-search/README.md (Twitter syndication, Reddit .json, HN Firebase, ...).
+```
+
+The full engine harness (rules R1-R7, the Phase 0 official-API index, the no-site-name rule, and the `references/insane-search/*.md` deep-dives for TLS, Playwright routing, Naver, media, etc.) is in [`references/insane-search/README.md`](references/insane-search/README.md). Read it before tuning the engine or adding a WAF profile.
+
+### Escalate to Tier 1.5 or Tier 2 when
+- The target is a Chinese / social platform with a native reader -> Tier 1.5.
+- insane-search returns empty/partial, or the page needs JS interaction, a screenshot, a persistent login, or media playback -> Tier 2.
+
+## Tier 1.5 — agent-reach (platform-native readers)
+
+**When**: the target is a platform with a first-class API/CLI that beats generic fetching — especially Chinese platforms that stealth browsers still cannot reach cleanly. Several channels are zero-config (Douyin, Weibo via Jina, V2EX, Reddit, Jina Reader, RSS, YouTube); others need a one-time auth you supply via environment variables if you have access.
+
+| Category | Platforms | Entry |
+|---|---|---|
+| social | xhs (Xiaohongshu), douyin, weibo, bilibili, V2EX, Reddit, Twitter/X | [references/agent-reach/social.md](references/agent-reach/social.md) |
+| web | Jina Reader, WeChat articles, RSS | [references/agent-reach/web.md](references/agent-reach/web.md) |
+| video | YouTube, Bilibili, podcast transcripts, Douyin video | [references/agent-reach/video.md](references/agent-reach/video.md) |
+| career | LinkedIn | [references/agent-reach/career.md](references/agent-reach/career.md) |
+| dev | GitHub (gh CLI) | [references/agent-reach/dev.md](references/agent-reach/dev.md) |
+| search | Exa AI | [references/agent-reach/search.md](references/agent-reach/search.md) |
+
+```bash
+mcporter call 'douyin.parse_douyin_video_info(url: "<URL>")'   # douyin, zero-config
+curl -s "https://r.jina.ai/https://weibo.com/<uid>/<pid>"      # weibo via Jina
+yt-dlp --dump-json "<bilibili-url>"                            # Bilibili (overseas: add --cookies-from-browser)
+curl -s "https://www.v2ex.com/api/topics/hot.json"            # V2EX public API
+```
+
+Routing table, per-platform auth (set `TWITTER_*` env vars, `gh auth login`, a transcription key — only if you have access), rate-limit notes, and known version quirks are in [references/agent-reach/README.md](references/agent-reach/README.md).
+
+## Tier 2 — Chrome stealth (real interaction)
+
+**When**: real interaction is needed (clicks, forms, screenshots, video, persistent login), or Tier 1/1.5 failed.
+
+CloakBrowser is a stealth Chromium with source-level fingerprint patches that passes Cloudflare Turnstile, FingerprintJS, BrowserScan, and 30+ detectors; agent-browser is the CDP automation CLI that drives it. Both are runtime-installed tools (not vendored here). Full setup, version pins, launch flow, cookie login, and cross-platform notes are in [references/chrome-stealth.md](references/chrome-stealth.md).
+
+```bash
+# 1. Launch CloakBrowser with CDP on :9242 (see chrome-stealth.md for install + venv).
+# 2. CloakBrowser launches tabless — open the first tab via CDP before any agent-browser command:
+curl -s -X PUT "http://127.0.0.1:9242/json/new?https://example.com"
+# 3. Drive it with agent-browser over CDP:
+agent-browser --cdp 9242 snapshot -i        # interactive elements (@eN refs)
+agent-browser --cdp 9242 click @e3
+agent-browser --cdp 9242 screenshot out.png
+agent-browser --cdp 9242 close
+```
+
+### Cookie login (cross-platform)
+
+`scripts/extract_cookies.py` reads cookies from a local Chromium-family or Firefox-family browser and optionally injects them into the running CDP session. It resolves browser profile paths and decrypts cookie values per-OS (macOS Keychain, Linux libsecret, Windows DPAPI):
+
+```bash
+# Extract cookies to a file:
+mkdir -p ~/.local/state/omo-cookies
+python3 scripts/extract_cookies.py --browser chrome --domain youtube.com --output ~/.local/state/omo-cookies/youtube.cookies.json
+# Extract and inject into the running CDP session:
+python3 scripts/extract_cookies.py --browser chrome --domain youtube.com --inject --cdp 9242
+```
+
+Cookie export files are written with owner-only `0600` permissions. Do not place live auth cookies in shared temp directories or commit them to a repo. Cookie injection sends values to CDP over stdin rather than argv. Cookies apply on next navigation — reload after injecting. Google services use fingerprint-bound tokens that may not transfer across browser profiles. Full detail in [references/chrome-stealth.md](references/chrome-stealth.md).
+
+## Reference docs
+
+| File | When to read |
+|------|-------------|
+| [references/insane-search/README.md](references/insane-search/README.md) | Tier-1 engine harness (R1-R7, Phase 0 API index, no-site-name rule) + its `*.md` deep-dives |
+| [references/agent-reach/README.md](references/agent-reach/README.md) | Tier-1.5 routing table, platform auth, per-category `*.md` |
+| [references/chrome-stealth.md](references/chrome-stealth.md) | Tier-2 CloakBrowser + agent-browser install, CDP flow, version pins, cookie login |
+
+## Environment variables
+
+```bash
+CLOAK_CDP_PORT=9242              # CloakBrowser CDP port (default 9242)
+AGENT_BROWSER_USER_AGENT="..."   # override UA to hide HeadlessChrome
+AGENT_BROWSER_HEADED=1           # show the browser window
+# agent-reach auth: set the channel-specific env vars from each tool's docs only if you have access
+# insane-search needs no env vars — it auto-installs deps on first run
+```
+
+## Anti-patterns
+
+- Do NOT launch Chrome stealth for plain text extraction — use Tier 1.
+- Do NOT pass an `--init-script` for the webdriver flag — CloakBrowser already patches it at source; the only required override is `--user-agent`.
+- Do NOT run agent-browser before creating the first tab via `curl -X PUT .../json/new` — CloakBrowser launches tabless.
+- Do NOT use vanilla Chrome when stealth is needed — always CloakBrowser.
+- Do NOT forget to `close` the session when done.
+- Do NOT inject cookies without reloading the page.
+- Do NOT hardcode site domains/selectors into `engine/**` or `waf_profiles.yaml` — runtime hints only (see the no-site-name rule in the insane-search reference).

package/dist/skills/ultimate-browsing/engine/__init__.py

@@ -0,0 +1,24 @@
+"""insane-search engine — generic WAF-profile-based fetch chain.
+
+No site-specific logic lives here. Site specifics belong to runtime hints or
+observations, never to code. See `../SKILL.md` for the No-Site-Name Rule.
+"""
+
+from .validators import Verdict, ValidationResult, validate, CHALLENGE_MARKERS
+from .waf_detector import detect
+from .url_transforms import TRANSFORMS, apply_transform
+from .fetch_chain import fetch
+from .result_schema import Attempt, FetchResult
+
+__all__ = [
+    "Verdict",
+    "ValidationResult",
+    "validate",
+    "CHALLENGE_MARKERS",
+    "detect",
+    "TRANSFORMS",
+    "apply_transform",
+    "fetch",
+    "FetchResult",
+    "Attempt",
+]

package/dist/skills/ultimate-browsing/engine/__main__.py

@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+"""CLI entrypoint for the insane-search engine.
+
+Usage:
+    python3 -m engine URL [--selector CSS] [--device auto|desktop|mobile]
+                          [--timeout N] [--max-attempts N] [--json] [--trace]
+
+Examples:
+    python3 -m engine "https://example.com/" --selector "h1"
+    python3 -m engine "https://example.com/" --json
+    python3 -m engine "https://example.com/" --device mobile --trace
+
+Exit codes:
+    0   strong_ok or weak_ok
+    1   ok=False (all attempts failed)
+    2   CLI arg error
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+
+from . import fetch
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(prog="python3 -m engine",
+                                description="Generic WAF-profile fetch chain.")
+    p.add_argument("url", help="URL to fetch.")
+    p.add_argument("--selector", "-s", action="append", default=None,
+                   dest="selectors", metavar="CSS",
+                   help="Positive-proof CSS selector. Repeatable.")
+    p.add_argument("--device", choices=("auto", "desktop", "mobile"), default="auto",
+                   help="Device class pin.")
+    p.add_argument("--timeout", type=int, default=25,
+                   help="Per-attempt timeout seconds (default 25).")
+    p.add_argument("--max-attempts", type=int, default=12,
+                   help="Upper bound across all phases (default 12).")
+    p.add_argument("--no-playwright", action="store_true",
+                   help="Skip Playwright fallback (curl-only).")
+    p.add_argument("--json", action="store_true",
+                   help="Emit FetchResult as JSON to stdout (content omitted).")
+    p.add_argument("--trace", action="store_true",
+                   help="Print per-attempt trace to stderr.")
+    return p
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    try:
+        result = fetch(
+            args.url,
+            success_selectors=args.selectors,
+            device_class=args.device,
+            timeout=args.timeout,
+            max_attempts=args.max_attempts,
+            enable_playwright=not args.no_playwright,
+        )
+    except Exception as e:
+        print(f"engine fatal: {type(e).__name__}: {e}", file=sys.stderr)
+        return 2
+
+    if args.trace:
+        print("=== trace ===", file=sys.stderr)
+        for att in result.trace:
+            d = att.to_dict()
+            imp = d.get("impersonate") or "-"
+            ref = d.get("referer") or "-"
+            print(
+                f"[{d['phase']:<8}] {d['executor']:<18} "
+                f"xform={d['url_transform']:<16} imp={imp:<14} ref={ref:<14} "
+                f"status={d['status']:>4} size={d['body_size']:>8} "
+                f"verdict={d['verdict']} {('err=' + d['error'][:60]) if d.get('error') else ''}",
+                file=sys.stderr,
+            )
+        print(f"=== summary: {result.summary} ===", file=sys.stderr)
+
+    # Surface R7 hint (API-first route) prominently when summary contains it,
+    # regardless of --trace flag — this is actionable guidance, not noise.
+    if "R7 API-first" in (result.summary or ""):
+        print(
+            "\n════════════════════════════════════════════════════════════════\n"
+            "⚠️  R7 triggered — consider API-first route instead of HTML grid.\n"
+            "   See summary below (or re-run with --trace for full attempt log).\n"
+            "════════════════════════════════════════════════════════════════",
+            file=sys.stderr,
+        )
+        # Also print the full summary (which includes the hint) so caller sees it.
+        print(result.summary, file=sys.stderr)
+
+    if args.json:
+        payload = result.to_dict()
+        print(json.dumps(payload, ensure_ascii=False, indent=2))
+    else:
+        # Default: HTML to stdout, status to stderr.
+        print(result.content, end="")
+        print(f"\n[engine] ok={result.ok} verdict={result.verdict} "
+              f"profile={result.profile_used} attempts={len(result.trace)}",
+              file=sys.stderr)
+
+    return 0 if result.ok else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

package/dist/skills/ultimate-browsing/engine/bias_check.py

@@ -0,0 +1,174 @@
+#!/usr/bin/env python3
+"""No-Site-Name Rule checker.
+
+Run in CI / pre-commit. Scans engine/** for hard-coded site names or
+domains that would bias the generic fetch chain toward one site.
+
+Exit code 0 if clean, 1 if violations found.
+
+    python3 engine/bias_check.py
+    python3 engine/bias_check.py --strict    # also check references/*.md (usually off)
+"""
+from __future__ import annotations
+
+import argparse
+import os
+import re
+import sys
+from pathlib import Path
+
+
+# Known brand / domain substrings that should NOT appear in engine code.
+# This is a non-exhaustive deny list. CI should treat hits as warnings that
+# require human review; false positives (e.g. "github" in comments) can be
+# whitelisted via EXPLICIT_ALLOW.
+BRAND_SUBSTRINGS = [
+    "coupang", "11st", "11번가", "musinsa", "무신사",
+    "fmkorea", "에펨코리아", "dcinside", "디시인사이드",
+    "ohou", "오늘의집", "kurly", "마켓컬리",
+    "daangn", "당근",
+    # Naver is allowed in Phase 0 references (official APIs) but not in engine code.
+    "naver.com", "blog.naver", "shopping.naver",
+    # Korean portal brand names
+    "daum.net", "kakao.com",
+]
+
+# Regex for bare URLs / domains. Used as a secondary pass to flag hardcoded
+# site hosts that slipped past the brand denylist.
+URL_PATTERN = re.compile(
+    r"https?://[\w\.-]+|[\w-]+\.(?:com|net|org|co\.kr|kr|io)\b",
+    re.IGNORECASE,
+)
+
+# Generic / neutral hosts that are allowed anywhere (examples, specs, stdlib,
+# and domains that legitimately appear as non-site-specific referrers / test
+# fixtures — Google search as a generic Referer strategy, httpbin for transport
+# tests, etc.). Anything in this set must be provably unrelated to a specific
+# target-site preference.
+URL_ALLOWLIST = {
+    "example.com", "example.org", "example.net",
+    "localhost", "127.0.0.1",
+    # Official API / documentation sources cited in code comments.
+    "curl.se", "playwright.dev", "nodejs.org", "npmjs.com",
+    # Generic Referer strategy target (used as a neutral off-site referer).
+    "www.google.com", "google.com",
+    # Generic HTTP test endpoint for infrastructure / transport tests.
+    "httpbin.org",
+}
+
+# Files / dirs that must be clean.
+SCAN_ROOTS_STRICT_OFF = ["engine"]
+SCAN_ROOTS_STRICT_ON = ["engine", "references"]
+
+# Directory names skipped during scan (third-party code, build artefacts).
+EXCLUDED_DIR_NAMES = {
+    "node_modules", "__pycache__", ".git", ".venv", "dist", "build",
+}
+
+# Comment markers within which a brand mention is OK (explanation).
+# Keyed per-extension; any line containing these is skipped.
+COMMENT_OK_MARKERS = {
+    ".py": ("# NOTE-BIAS-OK", "# EXAMPLE-ONLY"),
+    ".js": ("// NOTE-BIAS-OK", "// EXAMPLE-ONLY"),
+    ".yaml": ("# NOTE-BIAS-OK", "# EXAMPLE-ONLY"),
+    ".yml": ("# NOTE-BIAS-OK", "# EXAMPLE-ONLY"),
+    ".md": ("<!-- NOTE-BIAS-OK -->", "<!-- EXAMPLE-ONLY -->"),
+}
+
+# File paths explicitly exempted (full match against relative path from scan root).
+EXPLICIT_ALLOW_FILES = {
+    # None right now — add if needed with justification.
+}
+
+
+def _line_is_exempt(line: str, ext: str) -> bool:
+    markers = COMMENT_OK_MARKERS.get(ext, ())
+    return any(m in line for m in markers)
+
+
+def _scan_file(path: Path, root: Path) -> list[str]:
+    """Return list of violation strings for this file."""
+    rel = path.relative_to(root.parent)
+    if str(rel) in EXPLICIT_ALLOW_FILES:
+        return []
+
+    ext = path.suffix.lower()
+    try:
+        text = path.read_text(encoding="utf-8", errors="ignore")
+    except Exception as e:
+        return [f"{rel}:0 — read error: {e}"]
+
+    violations: list[str] = []
+    for lineno, line in enumerate(text.splitlines(), start=1):
+        if _line_is_exempt(line, ext):
+            continue
+        lowered = line.lower()
+        # 1) Brand / domain denylist
+        hit_brand = None
+        for brand in BRAND_SUBSTRINGS:
+            if brand.lower() in lowered:
+                hit_brand = brand
+                break
+        if hit_brand:
+            violations.append(f"{rel}:{lineno} — brand `{hit_brand}` in: {line.strip()[:120]}")
+            continue  # one violation per line
+        # 2) URL/domain regex scan — catches hosts that aren't in the denylist.
+        for match in URL_PATTERN.finditer(line):
+            host = match.group(0).lower()
+            host = host.split("//", 1)[-1].split("/", 1)[0]
+            if host in URL_ALLOWLIST:
+                continue
+            if host.endswith(".example.com") or host.endswith(".example.org"):
+                continue
+            violations.append(f"{rel}:{lineno} — hardcoded host `{host}` in: {line.strip()[:120]}")
+            break
+    return violations
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description="Scan engine for site-name bias")
+    parser.add_argument("--strict", action="store_true",
+                        help="Also scan references/*.md (usually noisy — off by default)")
+    parser.add_argument("--root", default=None,
+                        help="Skill root directory. Defaults to parent of this file.")
+    args = parser.parse_args(argv)
+
+    skill_root = Path(args.root) if args.root else Path(__file__).parent.parent
+    scan_roots = SCAN_ROOTS_STRICT_ON if args.strict else SCAN_ROOTS_STRICT_OFF
+
+    total_violations: list[str] = []
+    scanned = 0
+    for name in scan_roots:
+        root = skill_root / name
+        if not root.exists():
+            continue
+        for dirpath, dirnames, filenames in os.walk(root):
+            # In-place filter so os.walk skips these subtrees.
+            dirnames[:] = [d for d in dirnames if d not in EXCLUDED_DIR_NAMES]
+            for fname in filenames:
+                p = Path(dirpath) / fname
+                if p.suffix.lower() not in (".py", ".js", ".yaml", ".yml", ".md", ".ts", ".mjs"):
+                    continue
+                if p.name == "bias_check.py":
+                    continue  # self-exempt (this file lists the brands)
+                scanned += 1
+                total_violations.extend(_scan_file(p, skill_root))
+
+    print(f"[bias-check] scanned {scanned} files under {skill_root}")
+    if total_violations:
+        print(f"[bias-check] ❌ {len(total_violations)} violation(s):")
+        for v in total_violations:
+            print(f"  - {v}")
+        print()
+        print("Fix options:")
+        print("  1) Remove the brand name (preferred)")
+        print("  2) If genuinely explanatory, add '# NOTE-BIAS-OK' on the same line")
+        print("  3) If this is a Phase 0 official API reference, move it to references/*.md and rerun without --strict")
+        return 1
+
+    print("[bias-check] ✅ clean")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

package/dist/skills/ultimate-browsing/engine/curl_probe.py

@@ -0,0 +1,91 @@
+from __future__ import annotations
+
+import time
+from typing import Iterable, Mapping, Protocol
+
+from .referers import REFERER_STRATEGIES
+from .result_schema import Attempt
+from .validators import Verdict, validate
+
+
+class _CookieItem(Protocol):
+    name: str
+    value: str
+
+
+class _CookieJar(Protocol):
+    jar: Iterable[_CookieItem]
+
+
+class ProbeResponse(Protocol):
+    status_code: int
+    text: str
+    url: str
+    cookies: _CookieJar | Mapping[str, str]
+
+
+def _curl_probe(
+    url: str, *, impersonate: str, referer: str, timeout: int = 20
+) -> tuple[ProbeResponse | None, str | None]:
+    try:
+        from curl_cffi import requests as cffi_requests
+    except ImportError:
+        return None, "curl_cffi not installed"
+
+    headers = {
+        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
+        "Accept-Language": "ko-KR,ko;q=0.9,en-US;q=0.8,en;q=0.7",
+    }
+    if referer:
+        headers["Referer"] = referer
+
+    try:
+        resp = cffi_requests.get(
+            url,
+            impersonate=impersonate,
+            headers=headers,
+            timeout=timeout,
+            allow_redirects=True,
+        )
+        return resp, None
+    except cffi_requests.exceptions.RequestException as e:
+        return None, f"{type(e).__name__}:{str(e)[:200]}"
+
+
+def run_attempt(
+    url: str,
+    *,
+    transform_name: str,
+    impersonate: str,
+    referer_name: str,
+    success_selectors: list[str] | None,
+    known_bad_sizes: list[int] | None,
+    timeout: int,
+    phase: str,
+) -> tuple[Attempt, ProbeResponse | None]:
+    referer_url = REFERER_STRATEGIES.get(referer_name, REFERER_STRATEGIES["none"])(url)
+    started_at = time.time()
+    resp, err = _curl_probe(url, impersonate=impersonate, referer=referer_url, timeout=timeout)
+    elapsed = round(time.time() - started_at, 3)
+
+    att = Attempt(
+        phase=phase,
+        executor="curl_cffi",
+        url=url,
+        url_transform=transform_name,
+        impersonate=impersonate,
+        referer=referer_name,
+        elapsed_s=elapsed,
+    )
+
+    if err or resp is None:
+        att.error = err or "no response"
+        att.verdict = Verdict.UNKNOWN.value
+        return att, None
+
+    vr = validate(resp, success_selectors=success_selectors, known_bad_sizes=known_bad_sizes)
+    att.status = vr.status
+    att.body_size = vr.body_size
+    att.verdict = vr.verdict.value
+    att.reasons = vr.reasons
+    return att, resp