dataapp-resource-proxy 0.1.0__tar.gz

dataapp_resource_proxy-0.1.0/PKG-INFO

@@ -0,0 +1,6 @@
+Metadata-Version: 2.5
+Name: dataapp-resource-proxy
+Version: 0.1.0
+Summary: Run-scoped client for managed browser, proxy and CAPTCHA resources
+Requires-Python: >=3.11
+Requires-Dist: httpx<1,>=0.28

dataapp_resource_proxy-0.1.0/README.md

@@ -0,0 +1,273 @@
+# DataApp Resource Proxy client
+
+This package acquires run-scoped DataOS browser and proxy resources. It does not
+contain supplier credentials, crawler code, or server components.
+
+The local candidate supports `captcha_types(type=..., refresh=True)` and generic
+`solve_captcha(type=..., task_variant=..., parameters=...)`. Query the directory
+before selecting new variants. The developer directory only returns variants
+whose Billing classifications have effective supplier-cost quotes. Its
+`pricing_version` identifies the price snapshot used for that decision, and
+`available_billing_types` lists the enabled classifications without exposing
+supplier prices. Queries do not submit solver tasks; explicit solves are
+metered. Structured answers are available through `CaptchaTask.data`.
+See [the RP task-directory contract](../../docs/resources/captcha-directory.md).
+These fields require matching service/Runtime bridge versions; updating this
+client alone does not upgrade a deployed platform.
+
+CAPTCHA directory results use the same short client cache as other resource
+catalogs. Pass `refresh=True` when the UI needs current availability. The
+directory is advisory: Resource Proxy performs the authoritative check again at
+submission, so manually supplied or stale parameters cannot enable an unpriced
+task. Older directory responses without availability metadata are rejected
+instead of being treated as unrestricted support.
+
+The current development line supports account-environment routing, regional
+resource receipts, and the `DEFERRED` resource-cost receipt contract.
+
+For local development against pre, only the developer's ordinary API Key is
+required. The client defaults to the `bazhuayu_pre` account, uses the verified
+domestic and overseas pre entries, and selects the resource node independently
+from the account and Billing environment.
+
+Managed proxies and CAPTCHA solves accrue resource costs. After the developer
+confirms that the local test may use these services, run the masked setup:
+
+```sh
+python -m dataapp_resource_proxy configure
+```
+
+The command reads the Key without displaying it and stores it outside the
+project in a mode-`600` per-user file. Obtain the Key from **Account Center >
+API Keys**: [pre](https://pre.bazhuayu.com/console/account-center/api-keys) for
+the default pre setup, or
+[production](https://www.bazhuayu.com/console/account-center/api-keys) for a
+production account. Never paste a Key into chat or package it with an App.
+Platform Runs use injected run identity and do not need a developer API Key.
+When local code runs inside a container, the host's per-user file is not visible
+automatically. Mount that one file read-only at the same path, or pass
+`DATAAPP_API_KEY` through the container's secret/environment mechanism. Do not
+copy the file into the image or build context.
+
+Configure the application process with the following variables:
+
+```sh
+DATAAPP_API_KEY=<developer-api-key>
+DATAOS_RESOURCE_READ_TIMEOUT_SECONDS=10
+```
+
+`DATAAPP_API_KEY` is the canonical process override. The deprecated
+`DATAOS_API_KEY` remains a compatibility fallback; conflicting values are
+rejected. The per-user setup file is loaded only for missing values, so an
+explicit process value takes precedence. `DATAOS_APP_ID` remains optional
+metadata and is not a prerequisite for local managed resources.
+
+Regional `usages()` and `resource_charges()` reads use a 10-second per-node
+deadline by default, so one slow node does not hide another node's result. Set
+`DATAOS_RESOURCE_READ_TIMEOUT_SECONDS` to a finite value greater than 0 and at
+most 30 seconds when a different local deadline is required. The
+`regional_read_timeout` argument to `ResourceProxy.from_env()` takes precedence
+over the environment value. Hosts can read the validated
+`resources.regional_read_timeout` property when applying their own bounded
+execution-shutdown guard.
+
+The access profiles `bazhuayu_overseas_pre` and `bazhuayu_overseas` select an
+overseas node while retaining the corresponding `bazhuayu_pre` or `bazhuayu`
+account and Billing identity. They are routing profiles, not separate accounts.
+`octoparse_pre` and `octoparse` remain distinct account environments for
+compatibility. A node being reachable or healthy does not prove that identity
+validation, supplier allocation, target-site access, or business-data
+extraction will succeed.
+
+The built-in pre endpoints are `https://pre-resource-proxy.bazhuayu.com:443`
+for domestic resources and
+`https://pre-resource-proxy-overseas.bazhuayu.com:443` for overseas resources.
+Certificate and hostname verification stay enabled. Explicit URL variables below
+can override these defaults for an isolated test or rollback.
+
+Production profiles (`bazhuayu`, `bazhuayu_overseas`, and `octoparse`) have no built-in node
+addresses until their public endpoints are verified. Configure one explicit
+single-node URL:
+
+```sh
+DATAOS_RESOURCE_PROXY_URL=https://resource-proxy.example.com
+```
+
+Or configure regional nodes independently:
+
+```sh
+DATAOS_RESOURCE_PROXY_DOMESTIC_URL=https://domestic-resource-proxy.example.com
+DATAOS_RESOURCE_PROXY_OVERSEAS_URL=https://overseas-resource-proxy.example.com
+```
+
+Explicit URL variables override automatic pre-node selection. Supplying any
+regional URL enables explicit regional mode, so an omitted region remains
+unconfigured. This permits an overseas-only setup without routing domestic work
+to the overseas node. `DATAOS_RESOURCE_PROXY_URL` keeps legacy single-node
+semantics and is never supplemented with an automatic overseas node.
+
+Then use the same resource API:
+
+```python
+from dataapp_resource_proxy import ResourceProxy
+
+async with ResourceProxy.from_env() as resources:
+    proxy = await resources.proxy(
+        country="US",
+        proxy_group="overseas_basic",
+    )
+    print(proxy.lease_id)
+```
+
+The API Key, returned proxy URL, and browser connection headers are secrets. Keep
+them in local environment configuration; do not log, package, or store them in
+application data. See `local-resource.env.example` for a value-free setup file.
+
+New managed proxy requests accept `auto`, `domestic_basic`, `overseas_basic`,
+or `overseas_enhanced`. `auto` uses the basic managed service. Supplier ordering
+and location or duration support come from Resource Proxy's published catalog;
+the client does not select suppliers. Existing handles may still report a legacy
+group value from historical leases and remain readable.
+
+Query the managed catalog before asking a user to choose a location. This is also
+the preferred flow for an AI-generated DataApp:
+
+Catalog methods require a node with the catalog API enabled. Older nodes can
+still issue leases but return HTTP 404 for catalog queries. Report that capability
+as unavailable; do not treat it as an empty geographic coverage list or silently
+allocate a paid lease to discover coverage. A client package upgrade does not
+upgrade the Resource Proxy server.
+
+```python
+async with ResourceProxy.from_env() as resources:
+    groups = await resources.proxy_groups(network_preference="overseas")
+    countries = await resources.proxy_locations(
+        proxy_group="overseas_basic",
+        network_preference="overseas",
+    )
+    states = await resources.proxy_locations(
+        proxy_group="overseas_basic",
+        country="US",
+    )
+    if not states["items"]:
+        raise RuntimeError("No state coverage is currently listed; check the catalog status")
+    proxy = await resources.proxy(
+        proxy_group="overseas_basic",
+        country="US",
+        state=states["items"][0]["code"],
+    )
+```
+
+Catalog results describe configured geographic coverage. They are not a promise
+of real-time supplier inventory, so a later lease request can still report that
+a route is temporarily unavailable. `ResourceProxy.proxy_location_hints()`
+returns a small non-authoritative list of popular countries for empty-state UI;
+it never replaces a failed catalog query. Catalog queries use a 30-second,
+per-client cache; pass `refresh=True` to bypass it.
+
+For a locally developed Code App, `DATAOS_APP_ID` and `DATAOS_APP_VERSION` are
+optional attribution metadata. The resource flow uses the ordinary platform
+`DATAAPP_API_KEY`; it does not create a separate billing Key.
+
+Inside a platform DataApp, obtain the same client from the Runtime so the
+DataHub backend can bind requests to the current Run and Attempt without
+forwarding the developer API key:
+
+```python
+from bazhuayu_dataapp import DataApp
+
+async with DataApp:
+    resources = DataApp.resources()
+    browser = await resources.browser(session_timeout=900)
+    ws_url, headers = await browser.connection()
+```
+
+Resource handles are owned by the client. Close a handle when it is no longer
+needed, or close the client to release all remaining handles and its execution.
+`ProxyResource.rotate_credentials()` rotates only the temporary DataOS gateway
+username/password. It does not claim to replace the supplier session or exit IP.
+Creation requests accept an `idempotency_key`; reuse the same value only when
+retrying the same logical creation after an uncertain transport outcome. A
+browser call also gets an allocation `slot_id`, so independent calls do not
+share a CDP session. By default the slot is derived from the idempotency key;
+reusing that key safely replays the same logical creation. If a slot is supplied
+explicitly, reuse it together with the key. One client rejects a second live
+handle for the same remote browser so closing one crawler cannot stop another's
+shared session. Proxy leases use the same single-live-handle rule.
+
+
+## Managed CAPTCHA resources
+
+The default SDK path is the platform-managed cloud API below. Supplier
+credentials stay in Resource Proxy. A native browser extension is an explicit
+advanced option for developers who supply and manage their own vendor account;
+it is not enabled by local API-Key setup.
+
+Managed CAPTCHA calls use the overseas Resource Proxy node by default while
+retaining the caller's Bazhuayu account and Billing environment. Creation and
+polling stay on that node. Pass the same explicit `network_preference` to both
+operations only when testing another configured node.
+
+The same execution can call `create_captcha_task`, `get_captcha_task`, or
+`solve_captcha`. For image recognition use `type="image_to_text"`, a one-line
+Base64 `image_body`, and optional `capmonster_module="botdetect"` (or another
+configured module). Module selection is a recognizer parameter, not webpage
+CAPTCHA detection. Reuse the same `idempotency_key` when retrying a submission;
+never automatically resubmit an unknown outcome with a new key.
+
+`CaptchaTask` returns `node_id/network`; use the creating node for all polls.
+Solutions are excluded from repr. A solver answer still requires validation by
+the target site and does not prove business acquisition success.
+
+`captcha_plugin_grant(allowed_types=["image_to_text"], ttl_seconds=300)` returns
+an adapter URL and a secret short-lived key for the explicitly managed extension.
+Other canonical types are `turnstile`, `recaptcha_v2`, and
+`recaptcha_v2_enterprise`; `allowed_hosts` can narrow widget task sites. This key
+cannot authenticate IP, Browser, model, or execution APIs. Its
+`protocol_version` is currently integer `1`. The extension must use the adapter
+for create/poll/credit checks, preserve `dataosRequestId` across retries, and
+return opaque `taskId` unchanged; changing an original extension's base URL alone
+is insufficient. There is no upstream vendor credential in the grant.
+
+## Local debug billing
+
+For opted-in local executions, the SDK logs one warning before the first
+managed IP lease request, CAPTCHA submission, plugin grant, or model request.
+IP traffic, CAPTCHA solves, and model requests can charge the developer
+account. Pod cost appears separately in cloud debug.
+
+```python
+async with ResourceProxy.from_env() as resources:
+    completion = await resources.chat_completion({
+        "model": "your-model-alias",
+        "messages": [{"role": "user", "content": "Hello"}],
+    })
+
+receipt = await resources.resource_charge()
+if receipt is not None:
+    print(receipt["state"], receipt["currency"], receipt["expected_amount"], receipt["actual_amount"])
+```
+
+Read `resource_charge()` during the execution or after it closes. Expected
+and actual amounts retain the service's decimal strings. Resource costs will be
+deducted from developer earnings at settlement, and the remaining earnings will be
+paid out after deduction. `DEFERRED` means the cost has been recorded for deduction;
+it does not confirm a completed deduction. An `actual_amount` of `None` means there
+is no confirmed wallet amount. `resource_charge()`
+returns `None` when managed billing is disabled for the execution or the
+server predates managed billing. A `None` receipt is an unavailable receipt
+payload, not proof that the resource was free or that no charge was recorded.
+
+## Managed model requests
+
+Use `chat_completion()` with the local client or with `DataApp.resources()`
+in a cloud Run. The helper sends one non-streaming request. Omit `stream` or
+set it to `False`. If your caller may retry after a transport error, pass a
+stable `idempotency_key` and reuse it for the same logical request. The
+helper generates a key when you omit it.
+
+The direct HTTP/SSE endpoint `/v1/model-proxy/chat/completions` uses the
+execution identity. Raw clients must send `Idempotency-Key` for managed
+billing and reuse it on retries. Replaying a dispatched key returns a
+conflict. See the [server contract](../../docs/resources/captcha-model.md)
+for the inline-image format, model alias policy, and incomplete usage semantics.

dataapp_resource_proxy-0.1.0/local-resource.env.example

@@ -0,0 +1,22 @@
+# Prefer `python -m dataapp_resource_proxy configure`: it prompts without echo,
+# stores this value outside the project, and applies private file permissions.
+# This environment variable is the explicit process override. Never commit a
+# real value, paste it into chat, or package it with a Data App.
+DATAAPP_API_KEY=replace-with-developer-api-key
+
+# Account Center > API Keys:
+# pre: https://pre.bazhuayu.com/console/account-center/api-keys
+# production: https://www.bazhuayu.com/console/account-center/api-keys
+
+# Local debug defaults to bazhuayu_pre. These access profiles are optional;
+# overseas profiles keep the same Bazhuayu account and Billing identity.
+# DATAAPP_ENVIRONMENT=bazhuayu_overseas_pre
+
+# Pre node addresses are built in. DATAOS_* URL variables remain expert
+# transport overrides for isolated tests and verified production deployments.
+# Uncomment explicit URLs only to override automatic pre-node selection or to
+# configure a verified production deployment. Supplying one regional URL leaves
+# the other region unconfigured.
+# DATAOS_RESOURCE_PROXY_URL=https://single-node.example.com
+# DATAOS_RESOURCE_PROXY_DOMESTIC_URL=https://domestic-node.example.com
+# DATAOS_RESOURCE_PROXY_OVERSEAS_URL=https://overseas-node.example.com

dataapp_resource_proxy-0.1.0/pyproject.toml

@@ -0,0 +1,33 @@
+[build-system]
+requires = ["hatchling>=1.27"]
+build-backend = "hatchling.build"
+
+[project]
+name = "dataapp-resource-proxy"
+version = "0.1.0"
+description = "Run-scoped client for managed browser, proxy and CAPTCHA resources"
+requires-python = ">=3.11"
+dependencies = ["httpx>=0.28,<1"]
+
+[dependency-groups]
+dev = ["pytest>=8,<10", "pytest-asyncio>=0.24,<2", "ruff>=0.9", "mypy>=1.13"]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/dataapp_resource_proxy"]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+asyncio_default_fixture_loop_scope = "function"
+testpaths = ["tests"]
+
+[tool.ruff]
+target-version = "py311"
+line-length = 100
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B"]
+
+[tool.mypy]
+python_version = "3.11"
+check_untyped_defs = true
+ignore_missing_imports = true

dataapp_resource_proxy-0.1.0/src/dataapp_resource_proxy/__init__.py

@@ -0,0 +1,15 @@
+"""Client for DataOS managed runtime resources."""
+
+from .client import ResourceProxy, ResourceTransport
+from .errors import ResourceProxyError
+from .handles import BrowserResource, CaptchaPluginGrant, CaptchaTask, ProxyResource
+
+__all__ = [
+    "BrowserResource",
+    "CaptchaTask",
+    "CaptchaPluginGrant",
+    "ProxyResource",
+    "ResourceProxy",
+    "ResourceProxyError",
+    "ResourceTransport",
+]

dataapp_resource_proxy-0.1.0/src/dataapp_resource_proxy/__main__.py

@@ -0,0 +1,65 @@
+"""Command-line helpers for local DataApp resource configuration."""
+
+from __future__ import annotations
+
+import argparse
+import getpass
+import os
+import warnings
+from collections.abc import Sequence
+
+from ._local_config import (
+    load_project_config,
+    resolve_config_environment,
+    save_local_config,
+)
+from .errors import ResourceProxyError
+
+
+def _configure() -> int:
+    try:
+        project = load_project_config(os.environ)
+        environment = resolve_config_environment(os.environ, project)
+    except ResourceProxyError as error:
+        print(f"配置错误：{error}")
+        return 2
+    if environment is None:
+        print(
+            "未找到当前项目的 .dataapp-sdk.json。请在下载的 SDK 项目根目录运行，"
+            "或为独立项目显式设置 DATAAPP_ENVIRONMENT。"
+        )
+        return 2
+    print("平台托管 IP 代理和验证码服务会按实际用量产生资源费用。")
+    if project is not None:
+        print(f"请从当前平台账户中心获取 API Key：{project.api_keys_url}")
+    answer = input("确认在本地调试中使用这些收费资源？[y/N] ").strip().lower()
+    if answer not in {"y", "yes"}:
+        print("未保存配置，也不会发起任何托管资源请求。")
+        return 1
+    try:
+        with warnings.catch_warnings():
+            warnings.simplefilter("error", getpass.GetPassWarning)
+            api_key = getpass.getpass("请粘贴账户中心的 API Key（输入不会显示）：").strip()
+    except (EOFError, KeyboardInterrupt, getpass.GetPassWarning):
+        print("当前终端无法安全隐藏输入；未保存配置。请在交互式终端重新运行。")
+        return 2
+    if not api_key:
+        print("API Key 不能为空；未保存配置。")
+        return 2
+    path = save_local_config(api_key, environment=environment)
+    print(f"配置已安全保存到 {path}，文件权限已设置为 600。")
+    return 0
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(prog="python -m dataapp_resource_proxy")
+    commands = parser.add_subparsers(dest="command", required=True)
+    commands.add_parser("configure", help="configure local managed resources securely")
+    options = parser.parse_args(argv)
+    if options.command == "configure":
+        return _configure()
+    return 2
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())