oh-my-opencode 4.19.4 → 5.0.0-beta.10

package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/AGENTS.md

@@ -0,0 +1,179 @@
+# ultimate-browsing/engine — Generic WAF-Profile Fetch Chain (Python)
+
+**Generated:** 2026-08-10 / 38d268995
+
+## UPSTREAM BASELINE AND VERSION POLICY
+
+**READ THIS BEFORE TOUCHING `engine/**` OR PROPOSING AN UPSTREAM SYNC.**
+
+This engine is NOT project-original code. It is a vendored-and-modified snapshot of
+[fivetaku/insane-search](https://github.com/fivetaku/insane-search), and the version
+we run on is a deliberate choice, not an accident of neglect.
+
+### The pin
+
+| Fact | Value |
+|---|---|
+| Upstream project | `https://github.com/fivetaku/insane-search` |
+| Vendoring commit | `a4e4ed797` (2026-06-21) `feat(ultimate-browsing): vendor insane-search engine (junk-excluded)` |
+| De-personalization | `4743199a5` (2026-06-21) |
+| Pinned upstream baseline | upstream state as of 2026-06-21, **pre-0.7.0** (0.7.0 is dated 2026-06-22) |
+| Re-vendors since | none — every later change here is ours |
+
+We intentionally track a **pinned baseline plus local divergence**, not upstream HEAD.
+There is no submodule and no automated drift check for this engine (unlike the
+`frontend` skill's upstream submodules): the vendored files ARE the source of truth,
+and upstream is a reference we port FROM, deliberately, file by file.
+
+### Why we do not blind-rebase onto upstream HEAD
+
+1. **Upstream reset its public history.** On 2026-08-06 upstream published a single
+   squashed commit, `019ee16 refactor: reset public history at 0.14.0`, discarding the
+   prior public history through 0.13.x. There is no upstream commit graph to rebase onto and no
+   way to cherry-pick an individual upstream change by sha — only whole-file diffing
+   against a moving HEAD.
+2. **Upstream 0.14.0 REMOVED capability.** The endpoint-mining / internal-API
+   auto-derivation / site-recipe subsystems that upstream carried publicly between
+   0.12.0 and 0.14.0 are gone from upstream HEAD. Syncing to HEAD is therefore not
+   strictly an upgrade: parts of it are a downgrade relative to the intermediate
+   versions, and none of it is recoverable from the reset history.
+3. **Our tree diverged on purpose.** The KEEP list below is functionality upstream
+   never had. A wholesale overwrite with upstream HEAD would silently delete it.
+4. **Different threat model.** Our engine ships inside a published npm package and a
+   public marketplace mirror, under a CI no-site-name gate and a de-personalization
+   deny-list. Upstream carries neither constraint, so upstream code is not
+   drop-in-shippable here.
+
+### KEEP — our divergences a re-vendor MUST NOT regress
+
+These exist only in our tree. Any upstream sync that removes or bypasses one of them
+is a regression, not an upgrade:
+
+- **Phase 2.5 surrogate retrieval** (`surrogate.py`, `surrogates.yaml`) — archive /
+  reader / proxy routes tried before paying for a browser spin-up, with per-entry
+  `last_verified` staleness handling and `--allow-proxy` gating.
+- **Provenance / trust contract** (`result_schema.py`) — `Provenance` and `Trust`
+  literals on every result, so a snapshot can never be reported as the live page.
+- **Surrogate dead-end validation** (L1.5 in `validators.py`) — interstitial titles and
+  AMP-style redirect stubs rejected instead of returned as content.
+- **The no-site-name rule and its CI gate** (`bias_check.py`) — zero hard-coded site
+  names, brands, or target domains in `engine/**`.
+- **Module split of the fetch chain** — `curl_probe` / `referers` / `url_transforms` /
+  `waf_detector` / `validators` / `executor` / `summary` as separate modules rather than
+  one monolith.
+- **The Python test suite** under `engine/tests/` with its HTML/JSON fixtures.
+- **De-personalization** — no personal absolute paths, no personal auth token literals,
+  no personal browser choice; enforced by `depersonalization-gate.test.ts`.
+- **Skill-level layering** — the engine is Tier 1 under a router that also owns Tier 1.5
+  (agent-reach) and Tier 2 (CloakBrowser + agent-browser). Upstream has no such tiering.
+
+### WANT — upstream improvements worth porting forward
+
+Our snapshot predates these; they are wanted, and each must be ported as a reviewed,
+site-agnostic change that preserves every KEEP item above. Port individually; never as
+a tree overwrite:
+
+- **Content quality**: dedicated markdown conversion of fetched HTML, main-content
+  extraction, PDF text extraction, and JSON-LD rescue when the HTML body is thin.
+- **Transient-failure retry** and **render-merge** of statically fetched HTML with the
+  browser-rendered DOM.
+- **Differential block classification** — distinguishing a bot-detection block from an
+  infrastructure or authentication failure, instead of collapsing both into `challenge`.
+- **Additional stealth fetch backends** beyond the current Playwright templates, and
+  additional WAF vendor profiles.
+- **Per-host route learning** — remembering which route succeeded for a host, with a TTL
+  and a bounded store. Must stay runtime state, never committed site knowledge (R4).
+- **Engine-level Phase 0 routing** — the official-public-API preference is currently only
+  a documented rule (R5) the agent can skip; upstream moved it into code so it cannot be
+  skipped. Worth adopting.
+
+### OUT OF SCOPE
+
+- **The removed upstream endpoint-mining / internal-API auto-derivation / site-recipe
+  subsystems.** They are absent from upstream HEAD and are not reconstructed here. They
+  also sit against R3/R4 and R7's anti-bias rule: discovered internal endpoints are
+  runtime findings, never committed engine knowledge.
+- **Any upstream code carrying site-specific selectors, domains, or brand names.** It
+  fails `bias_check.py` at the door; re-derive it site-agnostically or leave it out.
+- **Automated upstream tracking.** No submodule, no drift check, no auto-bump. Syncing is
+  a deliberate, reviewed, human-initiated act.
+
+### THE SYNC RULE
+
+Any future upstream sync preserves BOTH sides. Concretely:
+
+1. Diff the specific upstream capability you want against our tree — do not overwrite
+   files wholesale, and never `git checkout` upstream over `engine/`.
+2. Port it as its own reviewed change, keeping every KEEP item intact.
+3. Re-run `python3 engine/bias_check.py` and the `engine/tests/` suite; a port that
+   introduces a site name or breaks a fixture does not ship.
+4. Update the pin table above (baseline, date, what was ported) in the same change, plus
+   the provenance section of [`../ATTRIBUTION.md`](../ATTRIBUTION.md).
+5. If a port must drop a KEEP item, say so explicitly in the PR and get it agreed first —
+   silent regressions of the KEEP list are the failure mode this policy exists to prevent.
+
+## OVERVIEW
+
+A 17-module Python package embedded in the `ultimate-browsing` skill: a site-agnostic fetch chain that escalates from a cheap curl probe to a real browser, with declarative WAF and surrogate registries. Not "optional scripts" — it has its own CLI entry (`python3 -m engine URL`), two YAML config schemas, a 4-file test suite, and a standalone CI guard. Package exports (`__init__.py`): `fetch`, `FetchResult`, `Attempt`, `Verdict`, `ValidationResult`, `validate`, `CHALLENGE_MARKERS`, `detect`, `TRANSFORMS`, `apply_transform`.
+
+## THE NO-SITE-NAME RULE (enforced in CI)
+
+`engine/**` must contain **zero** hard-coded site names, brands, or target domains. Site specifics belong to runtime hints or observations, never to code. `bias_check.py` is a standalone scanner enforcing this: a brand denylist, a URL regex scan, an allowlist for genuine infrastructure hosts (archive.org, r.jina.ai, google.com, httpbin.org, relay.invalid), and a `# NOTE-BIAS-OK` comment convention for legitimate exemptions such as test fixtures.
+
+```bash
+python3 engine/bias_check.py       # fails on any site-specific leak
+```
+
+## FETCH CHAIN PHASES
+
+```
+fetch(url, ...)                                   # fetch_chain.py
+  Phase 1  curl_probe.py    — curl_cffi TLS-impersonation probe
+  Phase 2  grid             — referer/transform/device attempt grid
+  Phase 2.5 surrogate.py    — third-party archive/reader/proxy routes
+  Phase 3  executor.py      — capability-matched Playwright fallback
+```
+
+Ordering is **not** hardcoded: each `waf_profiles.yaml` profile carries a `fallback_when_challenge` list that drives the ladder. `surrogate_wayback` precedes browser executors in every profile, so archives are tried before paying for a browser spin-up.
+
+## PROVENANCE / TRUST CONTRACT
+
+`result_schema.py` puts two literals on every `FetchResult`:
+
+- `Provenance = "live" | "snapshot" | "proxy"`
+- `Trust = "origin" | "archive" | "untrusted"`
+
+A `snapshot` result carries `snapshot_timestamp` and **must** be cited with that timestamp — never presented as the live page. `surrogates.yaml` `kind` fixes these values: `archive` -> snapshot/archive, `reader` -> live, `proxy` -> proxy/untrusted.
+
+## SURROGATE REGISTRY (`surrogates.yaml`)
+
+Site-agnostic infrastructure only. Every entry carries `last_verified` (ISO date); entries older than 90 days are deprioritized and flagged, because surrogate routes rot (a 2026-08 probe found 4 of 6 known routes dead or stubbed). `proxy` routes are MITM by construction: they require the explicit `--allow-proxy` flag and never receive `Cookie` or `Authorization` headers. Every surrogate response is re-validated with `target_url` set, so an interstitial or a redirect stub is rejected instead of returned as content.
+
+## VALIDATOR LAYERS (`validators.py`)
+
+```
+L1    challenge markers (CHALLENGE_MARKERS)
+L1.5  surrogate dead ends — interstitial titles + AMP-style redirect stubs
+      (is_redirect_stub(), needs target_url)
+L2    size/shape fingerprints
+L3+   content checks
+```
+
+## CLI
+
+```bash
+python3 -m engine URL [--selector S] [--device auto|desktop|mobile]
+                      [--timeout 25] [--max-attempts 12]
+                      [--no-playwright] [--allow-proxy] [--json] [--trace]
+```
+
+## TESTS
+
+`tests/` — `test_surrogate.py` (staleness, proxy gating, short-circuit), `test_surrogate_validators.py`, `test_fetch_chain.py`, `test_playwright_templates.py`, plus HTML/JSON fixtures under `tests/fixtures/`.
+
+## NOTES
+
+- `summary.py` emits an **R7 API-first hint** after >=3 challenge verdicts against a known WAF profile: look for `/api/`, `/graphql`, or `.json` endpoints, which usually carry weaker WAF protection than the HTML surface.
+- `templates/` holds the Playwright JS templates (`playwright_real_chrome.js`, `playwright_mobile_chrome.js`) the executor drives.
+- `url_transforms.py` transforms stay domain-agnostic (`mobile_subdomain`, `am_prefix`, `drop_www`).
+- Parent: [`packages/shared-skills/AGENTS.md`](../../../AGENTS.md).

package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/__main__.py

@@ -39,6 +39,10 @@ def build_parser() -> argparse.ArgumentParser:
                    help="Upper bound across all phases (default 12).")
     p.add_argument("--no-playwright", action="store_true",
                    help="Skip Playwright fallback (curl-only).")
+    p.add_argument("--allow-proxy", action="store_true",
+                   help=("Enable kind=proxy surrogate entries (raw relay routes; "
+                        "MITM by construction — never citable alone, never "
+                        "receives cookies/auth headers). Off by default."))
     p.add_argument("--json", action="store_true",
                    help="Emit FetchResult as JSON to stdout (content omitted).")
     p.add_argument("--trace", action="store_true",
@@ -56,6 +60,7 @@ def main(argv: list[str] | None = None) -> int:
             timeout=args.timeout,
             max_attempts=args.max_attempts,
             enable_playwright=not args.no_playwright,
+            allow_surrogate_proxy=args.allow_proxy,
         )
     except Exception as e:
         print(f"engine fatal: {type(e).__name__}: {e}", file=sys.stderr)
@@ -96,7 +101,9 @@ def main(argv: list[str] | None = None) -> int:
         # Default: HTML to stdout, status to stderr.
         print(result.content, end="")
         print(f"\n[engine] ok={result.ok} verdict={result.verdict} "
-              f"profile={result.profile_used} attempts={len(result.trace)}",
+              f"profile={result.profile_used} attempts={len(result.trace)} "
+              f"provenance={result.provenance}"
+              + (f" snapshot_ts={result.snapshot_timestamp}" if result.snapshot_timestamp else ""),
               file=sys.stderr)
 
     return 0 if result.ok else 1

package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/bias_check.py

@@ -54,6 +54,14 @@ URL_ALLOWLIST = {
     "www.google.com", "google.com",
     # Generic HTTP test endpoint for infrastructure / transport tests.
     "httpbin.org",
+    # Surrogate-tier infrastructure (engine/surrogates.yaml): archives/relays
+    # that serve a copy of ANY target URL. Same no-site-preference category as
+    # the google.com Referer — they are not target sites.
+    "archive.org", "web.archive.org",
+    "r.jina.ai",
+    # Placeholder host in the proxy-entry shape example; "canonical non-routable
+    # doc domain" (.invalid TLD), can never be a real target.
+    "relay.invalid",
 }
 
 # Files / dirs that must be clean.
@@ -80,6 +88,9 @@ EXPLICIT_ALLOW_FILES = {
     # None right now — add if needed with justification.
 }
 
+# NOTE-BIAS-OK convention for surrogate tests: fixture URLs (stub/interstitial
+# captures from third-party services) carry the marker on the flagged line.
+
 
 def _line_is_exempt(line: str, ext: str) -> bool:
     markers = COMMENT_OK_MARKERS.get(ext, ())

package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/fetch_chain.py

@@ -40,6 +40,7 @@ def fetch(
     timeout: int = 25,
     max_attempts: int = 12,
     enable_playwright: bool = True,   # hook left for executor module
+    allow_surrogate_proxy: bool = False,  # opt-in: kind=proxy registry entries
 ) -> FetchResult:
     """Fetch `url` using the generic grid.
 
@@ -175,73 +176,110 @@ def fetch(
                     if att.verdict in (Verdict.STRONG_OK.value, Verdict.WEAK_OK.value):
                         return _build_result(resp, att, trace, profile_used=profile_id)
 
-    # -------- Phase 3: Playwright fallback (profile-driven order) -----------
-    if enable_playwright:
-        try:
-            from .executor import run_playwright_fallback  # lazy import
-            # Honour profile's `fallback_when_challenge` list — iterate the
-            # caller-declared order instead of capability-inferred single pick.
-            fb_profile = load_profile(profile_used or "unknown_challenge", profiles=profiles)
-            fb_order = fb_profile.get("fallback_when_challenge") or ["playwright_real_chrome"]
-            pw_attempt = None
-            pw_content = ""
-            for fb_name in fb_order:
-                if fb_name == "curl_grid_exhaust":
-                    # Already performed in Phase 2; nothing more to do here.
-                    continue
-                pw_attempt, pw_content = run_playwright_fallback(
+    # -------- Phase 2.5 + 3: fallback ladder (profile-driven order) ---------
+    # Surrogate routes are NOT browser work, so they run even when the caller
+    # disabled Playwright; only browser executors honour `enable_playwright`.
+    try:
+        # Honour profile's `fallback_when_challenge` list — iterate the
+        # caller-declared order instead of capability-inferred single pick.
+        fb_profile = load_profile(profile_used or "unknown_challenge", profiles=profiles)
+        fb_order = fb_profile.get("fallback_when_challenge") or ["playwright_real_chrome"]
+        pw_attempt = None
+        pw_content = ""
+        for fb_name in fb_order:
+            if fb_name == "curl_grid_exhaust":
+                # Already performed in Phase 2; nothing more to do here.
+                continue
+            if fb_name.startswith("surrogate_"):
+                # Phase 2.5: archive/reader/proxy routes serving a copy of
+                # the target. Cheaper than a browser; provenance-labeled.
+                from . import surrogate as surrogate_mod
+                s_atts, s_meta, s_content = surrogate_mod.run_surrogate(
                     url,
-                    profile_id=profile_used or "unknown_challenge",
+                    registry=surrogate_mod.load_surrogates(),
+                    allow_proxy=allow_surrogate_proxy,
+                    timeout=timeout,
                     success_selectors=success_selectors,
-                    device_class=device_class,
-                    force_executor=fb_name,
                 )
-                trace.append(pw_attempt)
-                if pw_attempt.verdict in (Verdict.STRONG_OK.value, Verdict.WEAK_OK.value):
+                trace.extend(s_atts)
+                s_att = s_atts[-1]
+                if s_att.verdict in (Verdict.STRONG_OK.value, Verdict.WEAK_OK.value):
                     return FetchResult(
                         ok=True,
-                        content=pw_content,
-                        final_url=pw_attempt.url,
-                        verdict=pw_attempt.verdict,
+                        content=s_content,
+                        final_url=s_att.url,
+                        verdict=s_att.verdict,
                         profile_used=profile_used,
                         trace=trace,
-                        summary=f"Playwright fallback succeeded via {fb_name}",
+                        summary=(
+                            f"surrogate {s_meta.get('surrogate')} succeeded"
+                            f" (provenance={s_meta.get('provenance')}"
+                            + (f" snapshot_ts={s_meta.get('snapshot_timestamp')})" if s_meta.get('snapshot_timestamp') else ")")
+                        ),
+                        provenance=str(s_meta.get("provenance", "live")),
+                        snapshot_timestamp=s_meta.get("snapshot_timestamp"),
+                        trust=str(s_meta.get("trust", "origin")),
                     )
-            # Synthesize a placeholder if no iteration ran (empty list).
-            if pw_attempt is None:
-                pw_attempt = Attempt(
-                    phase="fallback",
-                    executor="none",
-                    url=url,
-                    url_transform="original",
-                    impersonate=None,
-                    referer="",
-                    verdict=Verdict.UNKNOWN.value,
-                    error="profile has empty fallback_when_challenge",
+                continue
+            if not enable_playwright:
+                continue
+            from .executor import run_playwright_fallback  # lazy import
+            pw_attempt, pw_content = run_playwright_fallback(
+                url,
+                profile_id=profile_used or "unknown_challenge",
+                success_selectors=success_selectors,
+                device_class=device_class,
+                force_executor=fb_name,
+            )
+            trace.append(pw_attempt)
+            if pw_attempt.verdict in (Verdict.STRONG_OK.value, Verdict.WEAK_OK.value):
+                return FetchResult(
+                    ok=True,
+                    content=pw_content,
+                    final_url=pw_attempt.url,
+                    verdict=pw_attempt.verdict,
+                    profile_used=profile_used,
+                    trace=trace,
+                    summary=f"Playwright fallback succeeded via {fb_name}",
                 )
-                trace.append(pw_attempt)
-        except ImportError:
-            trace.append(Attempt(
+        # Synthesize a placeholder only when the profile genuinely offers no
+        # fallback route. Entries skipped because the caller disabled the
+        # browser are a caller choice, not a profile defect — no trace noise.
+        actionable = [n for n in fb_order if n != "curl_grid_exhaust"]
+        if pw_attempt is None and not actionable:
+            pw_attempt = Attempt(
                 phase="fallback",
-                executor="playwright",
+                executor="none",
                 url=url,
                 url_transform="original",
                 impersonate=None,
                 referer="",
                 verdict=Verdict.UNKNOWN.value,
-                error="executor module not available",
-            ))
-        except (RuntimeError, OSError) as e:
-            trace.append(Attempt(
-                phase="fallback",
-                executor="playwright",
-                url=url,
-                url_transform="original",
-                impersonate=None,
-                referer="",
-                verdict=Verdict.UNKNOWN.value,
-                error=f"{type(e).__name__}:{str(e)[:200]}",
-            ))
+                error="profile has empty fallback_when_challenge",
+            )
+            trace.append(pw_attempt)
+    except ImportError:
+        trace.append(Attempt(
+            phase="fallback",
+            executor="playwright",
+            url=url,
+            url_transform="original",
+            impersonate=None,
+            referer="",
+            verdict=Verdict.UNKNOWN.value,
+            error="executor module not available",
+        ))
+    except (RuntimeError, OSError) as e:
+        trace.append(Attempt(
+            phase="fallback",
+            executor="playwright",
+            url=url,
+            url_transform="original",
+            impersonate=None,
+            referer="",
+            verdict=Verdict.UNKNOWN.value,
+            error=f"{type(e).__name__}:{str(e)[:200]}",
+        ))
 
     # -------- Give up, return best we have ----------------------------------
     summary = format_summary(trace, profile_used)

package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/result_schema.py

@@ -1,7 +1,10 @@
 from __future__ import annotations
 
 from dataclasses import asdict, dataclass, field
-from typing import Optional
+from typing import Literal, Optional
+
+Provenance = Literal["live", "snapshot", "proxy"]
+Trust = Literal["origin", "archive", "untrusted"]
 
 
 @dataclass
@@ -32,6 +35,9 @@ class FetchResult:
     profile_used: Optional[str] = None
     trace: list[Attempt] = field(default_factory=list)
     summary: str = ""
+    provenance: str = "live"                       # live | snapshot | proxy
+    snapshot_timestamp: Optional[str] = None      # archive's own timestamp, when snapshot
+    trust: str = "origin"                         # origin | archive | untrusted
 
     def to_dict(self) -> dict:
         return {
@@ -41,5 +47,8 @@ class FetchResult:
             "profile_used": self.profile_used,
             "trace": [a.to_dict() for a in self.trace],
             "summary": self.summary,
+            "provenance": self.provenance,
+            "snapshot_timestamp": self.snapshot_timestamp,
+            "trust": self.trust,
             "content_length": len(self.content),
         }

package/packages/omo-codex/plugin/skills/ultimate-browsing/engine/surrogate.py

@@ -0,0 +1,214 @@
+"""Phase 2.5 surrogate retrieval: archive / reader / proxy routes for blocked origins.
+
+When the curl grid cannot fetch the live page, this module reads a declarative
+registry (engine/surrogates.yaml) and tries third-party routes that serve a
+copy of the target instead. Provenance semantics are contractual:
+
+    kind=archive -> provenance="snapshot" (+ archive's own timestamp), trust=archive
+    kind=reader  -> provenance="live"     (server-side re-render of the live page)
+    kind=proxy   -> provenance="proxy", trust=untrusted; requires allow_proxy=True
+                    and never sends Cookie/Authorization headers (relay = MITM)
+
+Every response body passes through validators.validate() with target_url set,
+so AMP-style redirect stubs and search-engine interstitials are rejected
+rather than recorded as wins. Entries whose `last_verified` is older than
+MAX_STALE_DAYS are tried after fresh ones (routes rot faster than they are
+maintained).
+"""
+from __future__ import annotations
+
+import json
+import os
+import time
+from datetime import date
+
+try:
+    import yaml
+except ImportError:
+    yaml = None
+
+from .result_schema import Attempt
+from .validators import Verdict, validate
+
+SURROGATES_PATH = os.path.join(os.path.dirname(__file__), "surrogates.yaml")
+MAX_STALE_DAYS = 90
+
+DEFAULT_SURROGATES: dict = {
+    "wayback": {
+        "kind": "archive",
+        "trust": "archive",
+        "enabled": True,
+        "last_verified": "2026-08-09",
+        "discovery": {
+            "url": "https://archive.org/wayback/available?url={target}",
+            "flag_pointer": "archived_snapshots.closest.available",
+            "snapshot_url_pointer": "archived_snapshots.closest.url",
+            "snapshot_timestamp_pointer": "archived_snapshots.closest.timestamp",
+        },
+        "min_body_bytes": 3072,
+    },
+}
+
+
+def load_surrogates(path: str = SURROGATES_PATH) -> dict:
+    if yaml is None:
+        return dict(DEFAULT_SURROGATES)
+    try:
+        with open(path, encoding="utf-8") as fh:
+            loaded = yaml.safe_load(fh) or {}
+    except (OSError, yaml.YAMLError):
+        return dict(DEFAULT_SURROGATES)
+    if not isinstance(loaded, dict):
+        return dict(DEFAULT_SURROGATES)
+    usable = {k: v for k, v in loaded.items() if isinstance(v, dict) and k}
+    return usable or dict(DEFAULT_SURROGATES)
+
+
+def is_stale(entry: dict, max_age_days: int = MAX_STALE_DAYS, today: date | None = None) -> bool:
+    verified = entry.get("last_verified")
+    if isinstance(verified, date):
+        parsed = verified
+    elif isinstance(verified, str):
+        try:
+            parsed = date.fromisoformat(verified)
+        except ValueError:
+            return True
+    else:
+        return True
+    return ((today or date.today()) - parsed).days > max_age_days
+
+
+def _http_get(url: str, *, headers: dict | None = None, timeout: int = 25):
+    try:
+        from curl_cffi import requests as cffi_requests
+    except ImportError as exc:
+        raise RuntimeError("curl_cffi not installed") from exc
+    return cffi_requests.get(
+        url,
+        impersonate="chrome",
+        headers=headers or {"Accept": "text/html,application/xhtml+xml,application/json;q=0.9,*/*;q=0.8"},
+        timeout=timeout,
+        allow_redirects=True,
+    )
+
+
+def _json_pointer(obj, dotted: str):
+    current = obj
+    for part in dotted.split("."):
+        if not isinstance(current, dict):
+            return None
+        current = current.get(part)
+    return current
+
+
+def _provenance_of(kind: str) -> str:
+    if kind == "archive":
+        return "snapshot"
+    if kind == "proxy":
+        return "proxy"
+    return "live"
+
+
+def _skip_reason(name: str, entry: dict, allow_proxy: bool) -> str | None:
+    kind = entry.get("kind", "archive")
+    if kind == "proxy" and not allow_proxy:
+        return "proxy_requires_allow_proxy_flag"
+    required_env = entry.get("enabled_env")
+    if required_env and not os.environ.get(str(required_env)):
+        return f"missing_env:{required_env}"
+    if entry.get("enabled") is False and kind != "proxy":
+        return "disabled"
+    return None
+
+
+def _attempt(name: str, url: str, verdict: Verdict, error=None, status: int = 0, size: int = 0, elapsed: float = 0.0) -> Attempt:
+    return Attempt(
+        phase="surrogate",
+        executor=f"surrogate_{name}",
+        url=url,
+        url_transform="original",
+        impersonate=None,
+        referer="",
+        status=status,
+        body_size=size,
+        verdict=verdict.value,
+        error=(str(error)[:200] if error else None),
+        elapsed_s=round(elapsed, 3),
+    )
+
+
+def _resolve_target(name: str, entry: dict, target: str, timeout: int) -> tuple[str | None, str | None]:
+    discovery = entry.get("discovery")
+    if not discovery:
+        host_rotation = entry.get("host_rotation") or [None]
+        host = host_rotation[0]
+        template = entry.get("fetch")
+        if not template or host is None and "{host}" in template:
+            return None, None
+        return template.format(host=host, target=target), None
+    discovery_url = discovery.get("url", "").format(target=target)
+    resp = _http_get(discovery_url, timeout=timeout)
+    payload = resp.json()
+    if discovery.get("flag_pointer") and not _json_pointer(payload, discovery["flag_pointer"]):
+        return None, None
+    snap_url = _json_pointer(payload, discovery.get("snapshot_url_pointer", ""))
+    timestamp = _json_pointer(payload, discovery.get("snapshot_timestamp_pointer", "")) or None
+    return snap_url, timestamp
+
+
+def run_surrogate(
+    target: str,
+    *,
+    registry: dict | None = None,
+    allow_proxy: bool = False,
+    timeout: int = 25,
+    success_selectors: list[str] | None = None,
+) -> tuple[list[Attempt], dict, str]:
+    registry = dict(registry) if registry else load_surrogates()
+    ordered = sorted(registry.items(), key=lambda kv: int(is_stale(kv[1])))
+    live_meta = {"provenance": "live", "snapshot_timestamp": None, "trust": "origin", "surrogate": None}
+    attempts: list[Attempt] = []
+
+    for name, entry in ordered:
+        kind = entry.get("kind", "archive")
+        reason = _skip_reason(name, entry, allow_proxy)
+        if reason:
+            attempts.append(_attempt(name, target, Verdict.UNKNOWN, error=f"skipped:{reason}"))
+            continue
+        started = time.time()
+        try:
+            fetch_url, timestamp = _resolve_target(name, entry, target, timeout)
+        except Exception as exc:
+            attempts.append(_attempt(name, target, Verdict.UNKNOWN, error=f"{type(exc).__name__}:{exc}", elapsed=time.time() - started))
+            continue
+        if not fetch_url:
+            attempts.append(_attempt(name, target, Verdict.UNKNOWN, error="no snapshot available", elapsed=time.time() - started))
+            continue
+        try:
+            resp = _http_get(fetch_url, timeout=timeout)
+        except Exception as exc:
+            attempts.append(_attempt(name, fetch_url, Verdict.UNKNOWN, error=f"{type(exc).__name__}:{exc}", elapsed=time.time() - started))
+            continue
+        body = getattr(resp, "text", "") or ""
+        status = int(getattr(resp, "status_code", 0) or 0)
+        min_bytes = int(entry.get("min_body_bytes", 3072))
+        att = _attempt(name, fetch_url, Verdict.UNKNOWN, status=status, size=len(body), elapsed=time.time() - started)
+        attempts.append(att)
+        if status >= 400 or len(body) < min_bytes:
+            att.error = f"surrogate_unusable:status={status},size={len(body)}"
+            continue
+        vr = validate(resp, success_selectors=success_selectors, target_url=target)
+        att.verdict = vr.verdict.value
+        att.reasons = vr.reasons
+        if vr.ok:
+            meta = {
+                "provenance": _provenance_of(kind),
+                "snapshot_timestamp": timestamp if kind == "archive" else None,
+                "trust": str(entry.get("trust", "archive")),
+                "surrogate": name,
+            }
+            return attempts, meta, body
+
+    if not attempts:
+        attempts.append(_attempt("none", target, Verdict.UNKNOWN, error="no surrogate entries"))
+    return attempts, live_meta, ""