@tiny-fish/hermes 0.1.2 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,6 +1,12 @@
1
1
  # TinyFish for Hermes Agent
2
2
 
3
- `tinyfish-hermes` is the first-party TinyFish plugin for [Hermes Agent](https://github.com/NousResearch/Hermes-Agent). It registers a `tinyfish` web provider that serves Hermes' web search and extract tools through the TinyFish Search and Fetch REST APIs, plus a `tinyfish` cloud browser provider for credit-gated remote browser sessions.
3
+ `tinyfish-hermes` is the first-party TinyFish plugin for [Hermes Agent](https://github.com/NousResearch/Hermes-Agent). It gives Hermes:
4
+
5
+ | Capability | How |
6
+ | ---------- | --- |
7
+ | Search, Fetch (free) | `tinyfish` web provider behind Hermes' `web_search` / `web_extract` |
8
+ | Browser (credits) | `tinyfish` cloud browser provider behind Hermes' browser tools |
9
+ | Agent (credits) | `tinyfish_agent` tool: goal-based automation on a real site |
4
10
 
5
11
  ## Install
6
12
 
@@ -19,7 +25,7 @@ hermes plugins install tinyfish-io/tinyfish-web-agent-integrations/hermes --enab
19
25
 
20
26
  ## Authentication
21
27
 
22
- The plugin authenticates with a TinyFish API key — create one at [agent.tinyfish.ai/api-keys](https://agent.tinyfish.ai/api-keys). Keys are resolved in order:
28
+ The plugin authenticates with a TinyFish API key — create one at [agent.tinyfish.ai/api-keys](https://agent.tinyfish.ai/api-keys?source=hermes). `hermes plugins install` and `hermes tinyfish setup` both prompt for it (hidden input); `setup` also verifies it with one free search. Keys are resolved in order:
23
29
 
24
30
  | Env var | Source |
25
31
  | ------- | ------ |
@@ -35,7 +41,7 @@ The plugin registers a `hermes tinyfish` command and an in-session `/tinyfish-st
35
41
  | `setup [--yes] [--api-key KEY] [--no-web-backend] [--live]` | Route web tools to TinyFish and save the API key |
36
42
  | `status [--json]` | Non-secret configuration status |
37
43
  | `doctor [--json] [--live] [--live-paid]` | Diagnostics; `--live` runs one real search+fetch, `--live-paid` a browser session per credit policy |
38
- | `credits {status,set,reset}` | Inspect or change `tinyfish.credit_policy.browser` |
44
+ | `credits {status,set,reset}` | Inspect or change `tinyfish.credit_policy.{browser,agent}` |
39
45
  | `browser {enable,disable,status}` | Toggle `browser.cloud_provider: tinyfish` |
40
46
  | `usage [--json]` | Wallet balance, auto-reload, and billing rates |
41
47
 
@@ -57,19 +63,20 @@ tinyfish:
57
63
  location: US # also: language, recency_minutes, after_date,
58
64
  domain_type: news # before_date, page, purpose
59
65
  fetch:
60
- format: markdown # default output format for extract
61
- ttl: 300 # also: per_url_timeout_ms, links, image_links
66
+ ttl: 300 # also: per_url_timeout_ms (default 90000), links, image_links
62
67
  ```
63
68
 
64
69
  ## Browser sessions
65
70
 
66
- Setting `browser.cloud_provider: tinyfish` routes Hermes' browser tools through TinyFish remote browser sessions. Each session consumes TinyFish credits, so sessions are policy-gated via `tinyfish.credit_policy.browser`:
71
+ Setting `browser.cloud_provider: tinyfish` routes Hermes' browser tools — the default Browser Use `browser_exec`, or the built-in `browser_*` tools with `browser.backend: off` — through TinyFish remote browser sessions. Sessions consume TinyFish credits, so they are policy-gated via `tinyfish.credit_policy.browser`; `tinyfish_agent` runs use `tinyfish.credit_policy.agent`:
67
72
 
68
73
  | Policy | Behavior |
69
74
  | ------ | -------- |
70
- | `request` (default) | Every session goes through Hermes' approval gate |
71
- | `allow` | Sessions start without per-session approval |
72
- | `deny` | Browser tools are blocked while routed to TinyFish |
75
+ | `request` (default) | Opening a session (or starting an agent run) goes through Hermes' approval gate; answer `[s]ession` to approve the rest of the session |
76
+ | `allow` | No approval prompt |
77
+ | `deny` | Blocked |
78
+
79
+ One-shot `hermes chat -q` runs skip Hermes' own browser cleanup at exit, so the plugin closes any TinyFish sessions still open when they finish.
73
80
 
74
81
  ```yaml
75
82
  browser:
@@ -91,6 +98,7 @@ When the `tinyfish` MCP server is also configured in Hermes (`mcp_servers.tinyfi
91
98
  python -m pip install -e . -r requirements-dev.txt # dev deps are version-bounded in requirements-dev.txt
92
99
  make lint
93
100
  make test
101
+ scripts/e2e.sh # real Hermes: validate, doctor, git install; live Search/Fetch with TINYFISH_API_KEY
94
102
  ```
95
103
 
96
104
  ## License
@@ -0,0 +1,11 @@
1
+ # TinyFish installed
2
+
3
+ **Next:** route Hermes' web tools to TinyFish and verify your key:
4
+
5
+ ```bash
6
+ hermes tinyfish setup
7
+ ```
8
+
9
+ No key yet? Create one at [agent.tinyfish.ai/api-keys](https://agent.tinyfish.ai/api-keys?source=hermes) — Search and Fetch are free. `setup` will ask for it.
10
+
11
+ Check anytime with `hermes tinyfish doctor --live` or `/tinyfish-status` in a session.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@tiny-fish/hermes",
3
- "version": "0.1.2",
3
+ "version": "0.2.0",
4
4
  "description": "TinyFish web provider plugin for Hermes Agent, delivered via npm for the TinyFish CLI. Installed by `tinyfish connect hermes`; not a JavaScript library.",
5
5
  "license": "MIT",
6
6
  "homepage": "https://docs.tinyfish.ai",
@@ -24,6 +24,7 @@
24
24
  "files": [
25
25
  "tinyfish_hermes/*.py",
26
26
  "plugin.yaml",
27
+ "after-install.md",
27
28
  "pyproject.toml",
28
29
  "__init__.py"
29
30
  ],
package/plugin.yaml CHANGED
@@ -1,13 +1,26 @@
1
1
  name: tinyfish
2
- version: 0.1.2
3
- description: "First-party TinyFish provider plugin for Hermes Agent: Search and Fetch over the TinyFish REST APIs with API-key auth, plus credit-gated Browser sessions."
2
+ version: 0.2.0
3
+ description: "First-party TinyFish plugin for Hermes Agent: free Search and Fetch, plus credit-gated Browser sessions and the tinyfish_agent web agent."
4
4
  author: TinyFish
5
5
  kind: backend
6
+ # 0.19 added the pre_tool_call approve directive; older Hermes would skip credit approval.
7
+ requires_hermes: ">=0.19"
8
+ # httpx ships with Hermes core; without this Hermes prompts for deps and leaves non-TTY installs disabled.
9
+ python_runtime: external
6
10
  provides_web_providers:
7
11
  - tinyfish
8
12
  provides_browser_providers:
9
13
  - tinyfish
14
+ provides_tools:
15
+ - tinyfish_agent
16
+ provides_hooks:
17
+ - pre_tool_call
18
+ - pre_llm_call
19
+ - on_session_finalize
10
20
  requires_env:
11
- - TINYFISH_API_KEY
21
+ - name: TINYFISH_API_KEY
22
+ description: "TinyFish API key (Search and Fetch are free)"
23
+ url: "https://agent.tinyfish.ai/api-keys?source=hermes"
24
+ secret: true
12
25
  homepage: "https://tinyfish.ai"
13
26
  license: MIT
package/pyproject.toml CHANGED
@@ -4,8 +4,8 @@ build-backend = "hatchling.build"
4
4
 
5
5
  [project]
6
6
  name = "tinyfish-hermes"
7
- version = "0.1.2"
8
- description = "TinyFish web provider plugin for Hermes Agent - Search and Fetch over the TinyFish REST APIs"
7
+ version = "0.2.0"
8
+ description = "TinyFish plugin for Hermes Agent - Search, Fetch, Browser, and the tinyfish_agent web agent"
9
9
  readme = "README.md"
10
10
  license = { text = "MIT" }
11
11
  requires-python = ">=3.10"
@@ -45,9 +45,10 @@ __version__ = _resolve_version()
45
45
  def register(ctx: Any) -> None:
46
46
  """Register TinyFish providers, lifecycle hooks, and CLI commands with Hermes."""
47
47
 
48
- from .browser_provider import TinyFishBrowserProvider
48
+ from .agent_tool import TINYFISH_AGENT_SCHEMA, tinyfish_agent
49
+ from .browser_provider import TinyFishBrowserProvider, shutdown_cleanup
49
50
  from .credit_policy import pre_tool_call_policy
50
- from .provider import TinyFishWebSearchProvider
51
+ from .provider import TinyFishWebSearchProvider, _api_key
51
52
  from .routing_context import routing_context_hook
52
53
  from .setup_cli import (
53
54
  dispatch_tinyfish_cli,
@@ -63,6 +64,17 @@ def register(ctx: Any) -> None:
63
64
  ctx.register_browser_provider(TinyFishBrowserProvider())
64
65
  ctx.register_hook("pre_tool_call", pre_tool_call_policy)
65
66
  ctx.register_hook("pre_llm_call", routing_context_hook)
67
+ ctx.register_hook("on_session_finalize", shutdown_cleanup)
68
+ # tinyfish_agent spends credits, so it also needs the pre_tool_call gate.
69
+ if hasattr(ctx, "register_hook") and hasattr(ctx, "register_tool"):
70
+ ctx.register_tool(
71
+ name="tinyfish_agent",
72
+ toolset="tinyfish",
73
+ schema=TINYFISH_AGENT_SCHEMA,
74
+ handler=tinyfish_agent,
75
+ check_fn=lambda: bool(_api_key()),
76
+ emoji="🐟",
77
+ )
66
78
 
67
79
  if hasattr(ctx, "register_cli_command"):
68
80
 
@@ -0,0 +1,124 @@
1
+ """``tinyfish_agent``: goal-based TinyFish web automation as a Hermes tool."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import time
7
+ from typing import Any
8
+
9
+ from . import rest_client
10
+ from .provider import MISSING_KEY_ERROR, _api_key, _is_interrupted
11
+
12
+ TERMINAL_STATUSES = frozenset({"COMPLETED", "FAILED", "CANCELLED"})
13
+ # Hermes' tool batch deadline is 420s; stop polling well before it.
14
+ POLL_BUDGET_SECONDS = 300.0
15
+ POLL_INTERVAL_SECONDS = 3.0
16
+ # Hermes' context pruner targets results above ~8k chars.
17
+ MAX_RESULT_CHARS = 12_000
18
+
19
+ TINYFISH_AGENT_SCHEMA: dict[str, Any] = {
20
+ "name": "tinyfish_agent",
21
+ "description": (
22
+ "Run a TinyFish web agent on a real website to complete a goal: navigate, "
23
+ "click, fill forms, and extract data across multiple pages. Use for "
24
+ "multi-step site tasks that web_search/web_extract cannot do. Consumes "
25
+ "TinyFish credits. Returns the agent's result as JSON."
26
+ ),
27
+ "parameters": {
28
+ "type": "object",
29
+ "properties": {
30
+ "url": {"type": "string", "description": "Website URL to start on."},
31
+ "goal": {
32
+ "type": "string",
33
+ "description": (
34
+ "Plain-language task, including what data to return and its "
35
+ "shape, e.g. 'Return the first 5 product names and prices as JSON'."
36
+ ),
37
+ },
38
+ "browser_profile": {
39
+ "type": "string",
40
+ "enum": ["lite", "stealth"],
41
+ "description": "Use 'stealth' for sites with bot protection.",
42
+ },
43
+ "proxy_country_code": {
44
+ "type": "string",
45
+ "enum": ["US", "GB", "CA", "DE", "FR", "JP", "AU"],
46
+ "description": "Route through a proxy in this country.",
47
+ },
48
+ },
49
+ "required": ["url", "goal"],
50
+ },
51
+ }
52
+
53
+
54
+ def _result(payload: dict[str, Any]) -> str:
55
+ text = json.dumps(payload, ensure_ascii=False)
56
+ if len(text) <= MAX_RESULT_CHARS:
57
+ return text
58
+ # Partial JSON text, not the original structure, under a key that says so.
59
+ full = json.dumps(payload.pop("result", None), ensure_ascii=False)
60
+ payload["truncated"] = True
61
+ cut = MAX_RESULT_CHARS - len(json.dumps({**payload, "result_preview": ""}))
62
+ while cut > 0:
63
+ text = json.dumps({**payload, "result_preview": full[:cut]}, ensure_ascii=False)
64
+ if len(text) <= MAX_RESULT_CHARS:
65
+ return text
66
+ cut -= len(text) - MAX_RESULT_CHARS
67
+ return json.dumps(payload, ensure_ascii=False)
68
+
69
+
70
+ def _summary(run: dict[str, Any]) -> dict[str, Any]:
71
+ keys = ("run_id", "status", "result", "error", "streaming_url")
72
+ return {key: run[key] for key in keys if run.get(key) is not None}
73
+
74
+
75
+ def tinyfish_agent(args: dict[str, Any], **_: Any) -> str:
76
+ api_key = _api_key()
77
+ if not api_key:
78
+ return json.dumps({"error": MISSING_KEY_ERROR})
79
+ url = str(args.get("url") or "").strip()
80
+ goal = str(args.get("goal") or "").strip()
81
+ if not url or not goal:
82
+ return json.dumps({"error": "tinyfish_agent needs both url and goal."})
83
+
84
+ try:
85
+ started = rest_client.start_run(
86
+ url,
87
+ goal,
88
+ api_key=api_key,
89
+ browser_profile=str(args.get("browser_profile") or "lite"),
90
+ proxy_country_code=args.get("proxy_country_code") or None,
91
+ )
92
+ except rest_client.TinyFishRestError as exc:
93
+ return json.dumps({"error": str(exc)})
94
+ run_id = str(started.get("run_id") or "")
95
+ if not run_id:
96
+ return json.dumps({"error": "TinyFish Agent did not return a run_id."})
97
+
98
+ deadline = time.monotonic() + POLL_BUDGET_SECONDS
99
+ run: dict[str, Any] = {"run_id": run_id, "status": "PENDING"}
100
+ while time.monotonic() < deadline:
101
+ if _is_interrupted():
102
+ try:
103
+ rest_client.cancel_run(run_id, api_key=api_key)
104
+ except rest_client.TinyFishRestError as exc:
105
+ # The run may still be billing; never claim it stopped.
106
+ error = f"Cancel failed, run may still be active: {exc}"
107
+ return json.dumps(
108
+ {"run_id": run_id, "status": "UNKNOWN", "error": error}
109
+ )
110
+ return json.dumps({"run_id": run_id, "status": "CANCELLED"})
111
+ time.sleep(POLL_INTERVAL_SECONDS)
112
+ try:
113
+ run = rest_client.get_run(run_id, api_key=api_key)
114
+ except rest_client.TinyFishRestError as exc:
115
+ # Retry transient failures; a 4xx (bar 429) will not fix itself.
116
+ if exc.status and 400 <= exc.status < 500 and exc.status != 429:
117
+ return json.dumps({"run_id": run_id, "error": str(exc)})
118
+ continue
119
+ if str(run.get("status") or "").upper() in TERMINAL_STATUSES:
120
+ return _result(_summary(run))
121
+
122
+ pending = _summary(run)
123
+ pending["note"] = "Still running on TinyFish after 300s; give the user the run_id."
124
+ return _result(pending)
@@ -15,7 +15,7 @@ except Exception: # pragma: no cover - lets the package import outside Hermes
15
15
 
16
16
 
17
17
  from . import rest_client
18
- from .config import credit_policy, tinyfish_config
18
+ from .config import browser_cloud_provider, credit_policy, tinyfish_config
19
19
  from .credit_policy import block_message
20
20
  from .provider import API_KEY_URL, MISSING_KEY_ERROR, _api_key
21
21
 
@@ -81,7 +81,8 @@ class TinyFishBrowserProvider(_HermesBrowserProvider): # type: ignore[misc]
81
81
  raise RuntimeError(
82
82
  "TinyFish Browser did not return session_id and cdp_url."
83
83
  )
84
- session_name = f"tinyfish_{task_id}_{uuid.uuid4().hex[:8]}"
84
+ # hermes_* lets Hermes' orphan reaper clean up after a crash.
85
+ session_name = f"hermes_tf_{uuid.uuid4().hex[:10]}"
85
86
  logger.info("Created TinyFish browser session")
86
87
  return {
87
88
  "session_name": session_name,
@@ -131,3 +132,17 @@ class TinyFishBrowserProvider(_HermesBrowserProvider): # type: ignore[misc]
131
132
  ],
132
133
  "post_setup": "agent_browser",
133
134
  }
135
+
136
+
137
+ def shutdown_cleanup(**kwargs: Any) -> None:
138
+ """``on_session_finalize`` hook: ``-q`` runs skip Hermes' atexit browser cleanup."""
139
+
140
+ # Other reasons (/new, idle) finalize one session while the process keeps others.
141
+ if kwargs.get("reason") != "shutdown" or browser_cloud_provider() != "tinyfish":
142
+ return
143
+ try:
144
+ from tools.browser_tool_lifecycle import cleanup_all_browsers
145
+
146
+ cleanup_all_browsers()
147
+ except Exception as exc:
148
+ logger.debug("TinyFish shutdown cleanup failed (%s)", type(exc).__name__)
@@ -4,10 +4,10 @@ from __future__ import annotations
4
4
 
5
5
  from typing import Any, Literal
6
6
 
7
- CreditFeature = Literal["browser"]
7
+ CreditFeature = Literal["browser", "agent"]
8
8
  CreditPolicy = Literal["deny", "request", "allow"]
9
9
 
10
- CREDIT_FEATURES: tuple[CreditFeature, ...] = ("browser",)
10
+ CREDIT_FEATURES: tuple[CreditFeature, ...] = ("browser", "agent")
11
11
  CREDIT_POLICIES: tuple[CreditPolicy, ...] = ("deny", "request", "allow")
12
12
  # 'request' is inert until browser.cloud_provider is tinyfish, and still
13
13
  # routes each session through Hermes approval once it is.
@@ -39,10 +39,13 @@ def tinyfish_config(config: dict[str, Any] | None = None) -> dict[str, Any]:
39
39
 
40
40
 
41
41
  def normalize_feature(value: str) -> CreditFeature:
42
- if value.strip().lower().replace("_", "-") == "browser":
42
+ feature = value.strip().lower().replace("_", "-")
43
+ if feature == "browser":
43
44
  return "browser"
45
+ if feature == "agent":
46
+ return "agent"
44
47
  raise ValueError(
45
- f"Unknown TinyFish credit feature '{value}'. Valid features: browser"
48
+ f"Unknown TinyFish credit feature '{value}'. Valid features: browser, agent"
46
49
  )
47
50
 
48
51
 
@@ -155,10 +158,3 @@ def fetch_options(config: dict[str, Any] | None = None) -> FetchOptions:
155
158
  if value is not None:
156
159
  options[key] = value
157
160
  return options
158
-
159
-
160
- def default_fetch_format(config: dict[str, Any] | None = None) -> str:
161
- section = tinyfish_config(config).get("fetch") or {}
162
- if isinstance(section, dict) and section.get("format"):
163
- return str(section["format"])
164
- return "markdown"
@@ -16,9 +16,15 @@ from .config import (
16
16
 
17
17
  FEATURE_LABELS: dict[CreditFeature, str] = {
18
18
  "browser": "TinyFish Browser",
19
+ "agent": "TinyFish Agent",
19
20
  }
20
21
 
21
22
 
23
+ def rule_key(feature: CreditFeature) -> str:
24
+ # Domain-free so one [s]ession answer covers every step of the session.
25
+ return f"tinyfish:{feature}"
26
+
27
+
22
28
  def target_domain(target: str | None) -> str:
23
29
  if not target:
24
30
  return ""
@@ -75,11 +81,8 @@ def request_credit_approval(
75
81
  try:
76
82
  from tools.approval import request_tool_approval
77
83
 
78
- # Domain-grained key: one [a]lways covers the browse; new domains still prompt.
79
84
  result = request_tool_approval(
80
- f"tinyfish_{normalized}",
81
- reason,
82
- rule_key=f"tinyfish:{normalized}:{target_domain(target)}",
85
+ f"tinyfish_{normalized}", reason, rule_key=rule_key(normalized)
83
86
  )
84
87
  except Exception as exc:
85
88
  return False, (
@@ -107,7 +110,7 @@ def _directive_for_feature(
107
110
  return {
108
111
  "action": "approve",
109
112
  "message": approval_reason(feature, operation, target),
110
- "rule_key": f"tinyfish:{feature}:{target_domain(target)}",
113
+ "rule_key": rule_key(feature),
111
114
  }
112
115
  return None
113
116
 
@@ -118,11 +121,14 @@ def pre_tool_call_policy(
118
121
  """Hermes plugin hook for policy-gating TinyFish credit-consuming tools."""
119
122
 
120
123
  params = args or {}
124
+ target = str(params.get("url") or params.get("target") or "")
121
125
 
126
+ if tool_name == "tinyfish_agent":
127
+ return _directive_for_feature("agent", tool_name, target)
122
128
  if not tool_name.startswith("browser_"):
123
129
  return None
124
130
  config = load_config()
125
131
  if browser_cloud_provider(config) != "tinyfish":
126
132
  return None
127
- target = str(params.get("url") or params.get("target") or "")
133
+ # Any browser_* call can open a billed session (first call, or after inactivity).
128
134
  return _directive_for_feature("browser", tool_name, target, config)
@@ -15,16 +15,16 @@ except Exception: # pragma: no cover - lets the package import outside Hermes
15
15
 
16
16
 
17
17
  from . import rest_client
18
- from .config import default_fetch_format, fetch_options, search_options
18
+ from .config import fetch_options, search_options
19
19
  from .normalize import normalize_fetch_documents, normalize_search_response
20
20
 
21
21
  logger = logging.getLogger(__name__)
22
22
 
23
23
  API_KEY_ENV_VARS = ("TINYFISH_API_KEY", "MCP_TINYFISH_API_KEY")
24
- API_KEY_URL = "https://agent.tinyfish.ai/api-keys"
24
+ API_KEY_URL = "https://agent.tinyfish.ai/api-keys?source=hermes"
25
25
  MISSING_KEY_ERROR = (
26
- "TinyFish API key not found. Set TINYFISH_API_KEY or MCP_TINYFISH_API_KEY; "
27
- f"create a key at {API_KEY_URL}."
26
+ f"TinyFish API key not found. Create one at {API_KEY_URL}, then run "
27
+ "`hermes tinyfish setup` (or set TINYFISH_API_KEY)."
28
28
  )
29
29
 
30
30
 
@@ -131,33 +131,18 @@ class TinyFishWebSearchProvider(_HermesWebSearchProvider): # type: ignore[misc]
131
131
  api_key = _api_key()
132
132
  if not api_key:
133
133
  return [_error_document(url, MISSING_KEY_ERROR) for url in urls]
134
- output_format = str(
135
- kwargs.get("format")
136
- or kwargs.get("output_format")
137
- or default_fetch_format()
138
- )
134
+ # Hermes caches extract results per requested format, so honour it verbatim.
135
+ output_format = str(kwargs.get("format") or "markdown")
139
136
  options = fetch_options()
140
- documents: list[dict[str, Any]] = []
141
- # The Fetch API caps a request at 10 URLs; oversized batches 400.
142
- for start in range(0, len(urls), rest_client.FETCH_MAX_URLS):
143
- if start and _is_interrupted():
144
- documents.extend(
145
- _error_document(url, "Interrupted") for url in urls[start:]
146
- )
147
- break
148
- chunk = urls[start : start + rest_client.FETCH_MAX_URLS]
149
- try:
150
- raw = rest_client.fetch(
151
- chunk,
152
- api_key=api_key,
153
- output_format=output_format,
154
- **options,
155
- )
156
- documents.extend(normalize_fetch_documents(raw, fallback_urls=chunk))
157
- except Exception as exc: # noqa: BLE001 - controlled failure envelope
158
- logger.warning("TinyFish REST fetch failed (%s)", type(exc).__name__)
159
- documents.extend(
160
- _error_document(url, _safe_rest_failure("fetch", exc))
161
- for url in chunk
162
- )
163
- return documents
137
+ # Hermes abandons a whole extract at 120s; per-URL timeouts keep the fast pages.
138
+ options.setdefault("per_url_timeout_ms", 90_000)
139
+ try:
140
+ raw = rest_client.fetch(
141
+ urls, api_key=api_key, output_format=output_format, **options
142
+ )
143
+ except Exception as exc: # noqa: BLE001 - controlled failure envelope
144
+ logger.warning("TinyFish REST fetch failed (%s)", type(exc).__name__)
145
+ return [
146
+ _error_document(url, _safe_rest_failure("fetch", exc)) for url in urls
147
+ ]
148
+ return normalize_fetch_documents(raw, fallback_urls=urls)
@@ -12,9 +12,9 @@ from . import __version__
12
12
 
13
13
  SEARCH_URL = "https://api.search.tinyfish.ai"
14
14
  FETCH_URL = "https://api.fetch.tinyfish.ai"
15
- FETCH_MAX_URLS = 10
16
15
  BROWSER_URL = "https://api.browser.tinyfish.ai"
17
- WALLET_URL = "https://agent.tinyfish.ai/v1/wallet"
16
+ AGENT_URL = "https://agent.tinyfish.ai"
17
+ WALLET_URL = f"{AGENT_URL}/v1/wallet"
18
18
  SEARCH_USAGE_URL = f"{SEARCH_URL}/usage"
19
19
  FETCH_USAGE_URL = f"{FETCH_URL}/usage"
20
20
 
@@ -27,6 +27,10 @@ _BROWSER_CLOSE_RETRY_BASE_SECONDS = 0.25
27
27
  class TinyFishRestError(RuntimeError):
28
28
  """Raised for TinyFish REST transport or HTTP failures."""
29
29
 
30
+ def __init__(self, message: str, status: int | None = None) -> None:
31
+ super().__init__(message)
32
+ self.status = status
33
+
30
34
 
31
35
  class TinyFishWalletNotFound(TinyFishRestError):
32
36
  """Raised when an account uses legacy billing or has no wallet yet."""
@@ -52,9 +56,11 @@ def _raise_http_error(service: str, exc: httpx.HTTPStatusError) -> NoReturn:
52
56
  status = exc.response.status_code
53
57
  if status == 402:
54
58
  raise TinyFishRestError(
55
- f"{service} returned HTTP 402. TinyFish credits or billing may be required."
59
+ f"{service} returned HTTP 402. "
60
+ "TinyFish credits or billing may be required.",
61
+ status,
56
62
  ) from exc
57
- raise TinyFishRestError(f"{service} returned HTTP {status}") from exc
63
+ raise TinyFishRestError(f"{service} returned HTTP {status}", status) from exc
58
64
 
59
65
 
60
66
  def search(
@@ -113,8 +119,8 @@ def fetch(
113
119
  image_links: bool | None = None,
114
120
  ttl: int | None = None,
115
121
  per_url_timeout_ms: int | None = None,
116
- # Batches can run ~120s server-side; docs recommend a client timeout >= 150s.
117
- timeout: float = 150.0,
122
+ # Hermes abandons an extract at 120s, so return our own error before that.
123
+ timeout: float = 110.0,
118
124
  ) -> dict[str, Any]:
119
125
  """Run the TinyFish Fetch API for one or more URLs."""
120
126
 
@@ -274,3 +280,63 @@ def wallet(*, api_key: str, timeout: float = 30.0) -> dict[str, Any]:
274
280
  raise TinyFishRestError(f"Could not reach TinyFish Wallet: {exc}") from exc
275
281
  except ValueError as exc:
276
282
  raise TinyFishRestError("TinyFish Wallet returned invalid JSON") from exc
283
+
284
+
285
+ def _agent_request(
286
+ method: str,
287
+ path: str,
288
+ *,
289
+ api_key: str,
290
+ json_body: dict[str, Any] | None = None,
291
+ timeout: float = 30.0,
292
+ ) -> dict[str, Any]:
293
+ try:
294
+ response = httpx.request(
295
+ method,
296
+ f"{AGENT_URL}{path}",
297
+ json=json_body,
298
+ headers=_json_headers(api_key),
299
+ timeout=timeout,
300
+ )
301
+ response.raise_for_status()
302
+ data = response.json()
303
+ except httpx.HTTPStatusError as exc:
304
+ _raise_http_error("TinyFish Agent", exc)
305
+ except httpx.RequestError as exc:
306
+ raise TinyFishRestError(f"Could not reach TinyFish Agent: {exc}") from exc
307
+ except ValueError as exc:
308
+ raise TinyFishRestError("TinyFish Agent returned invalid JSON") from exc
309
+ if not isinstance(data, dict):
310
+ raise TinyFishRestError("TinyFish Agent returned an unexpected response")
311
+ return data
312
+
313
+
314
+ def start_run(
315
+ url: str,
316
+ goal: str,
317
+ *,
318
+ api_key: str,
319
+ browser_profile: str = "lite",
320
+ proxy_country_code: str | None = None,
321
+ ) -> dict[str, Any]:
322
+ """Enqueue a TinyFish Agent run; returns immediately with ``run_id``."""
323
+
324
+ body: dict[str, Any] = {
325
+ "url": url,
326
+ "goal": goal,
327
+ "browser_profile": browser_profile,
328
+ "api_integration": "hermes",
329
+ }
330
+ if proxy_country_code:
331
+ body["proxy_config"] = {"enabled": True, "country_code": proxy_country_code}
332
+ return _agent_request(
333
+ "POST", "/v1/automation/run-async", api_key=api_key, json_body=body
334
+ )
335
+
336
+
337
+ def get_run(run_id: str, *, api_key: str) -> dict[str, Any]:
338
+ return _agent_request("GET", f"/v1/runs/{run_id}", api_key=api_key)
339
+
340
+
341
+ def cancel_run(run_id: str, *, api_key: str) -> dict[str, Any]:
342
+ return _agent_request("POST", f"/v1/runs/{run_id}/cancel", api_key=api_key)
@@ -6,7 +6,8 @@ from collections.abc import Mapping, Sequence
6
6
  from typing import Any
7
7
  from urllib.parse import urlparse
8
8
 
9
- from .config import load_config, routing_context_enabled
9
+ from .config import browser_cloud_provider, load_config, routing_context_enabled
10
+ from .provider import API_KEY_URL, _api_key
10
11
 
11
12
  ROUTING_CONTEXT_MARKER = '<tinyfish-routing-context version="1">'
12
13
  ROUTING_GUIDANCE = f"""{ROUTING_CONTEXT_MARKER}
@@ -16,6 +17,18 @@ TinyFish tool-routing guidance:
16
17
  - Infer the choice from the user's plain language. Do not ask them to choose MCP versus the provider, and do not persist per-request controls as configuration. If a required native tool is unavailable, use the generic provider only when it can preserve the requested constraints; otherwise explain which control is unavailable rather than silently dropping it."""
17
18
 
18
19
 
20
+ SETUP_CONTEXT_MARKER = '<tinyfish-setup-context version="1">'
21
+ SETUP_GUIDANCE = f"""{SETUP_CONTEXT_MARKER}
22
+ TinyFish is installed but has no API key, so its web search, fetch, and browser tools are unavailable. If the user needs web access, tell them to create a free key at {API_KEY_URL} and run `hermes tinyfish setup`."""
23
+
24
+
25
+ def tinyfish_backend_configured(config: dict[str, Any]) -> bool:
26
+ web = config.get("web") or {}
27
+ web_keys = ("search_backend", "extract_backend", "backend")
28
+ routed = isinstance(web, dict) and "tinyfish" in {web.get(k) for k in web_keys}
29
+ return routed or browser_cloud_provider(config) == "tinyfish"
30
+
31
+
19
32
  def tinyfish_mcp_configured(config: dict[str, Any]) -> bool:
20
33
  servers = config.get("mcp_servers") or {}
21
34
  if not isinstance(servers, dict):
@@ -32,18 +45,20 @@ def tinyfish_mcp_configured(config: dict[str, Any]) -> bool:
32
45
  )
33
46
 
34
47
 
35
- def _contains_routing_marker(value: object) -> bool:
48
+ def _contains_marker(value: object, marker: str) -> bool:
36
49
  if isinstance(value, str):
37
- return ROUTING_CONTEXT_MARKER in value
50
+ return marker in value
38
51
  if isinstance(value, Mapping):
39
- return any(_contains_routing_marker(item) for item in value.values())
52
+ return any(_contains_marker(item, marker) for item in value.values())
40
53
  if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
41
- return any(_contains_routing_marker(item) for item in value)
54
+ return any(_contains_marker(item, marker) for item in value)
42
55
  return False
43
56
 
44
57
 
45
- def routing_guidance_present(conversation_history: object) -> bool:
46
- """Return whether Hermes already carries this routing version in context."""
58
+ def routing_guidance_present(
59
+ conversation_history: object, marker: str = ROUTING_CONTEXT_MARKER
60
+ ) -> bool:
61
+ """Return whether Hermes already carries this marker version in context."""
47
62
 
48
63
  if not isinstance(conversation_history, Sequence) or isinstance(
49
64
  conversation_history, (str, bytes, bytearray)
@@ -52,9 +67,9 @@ def routing_guidance_present(conversation_history: object) -> bool:
52
67
  for message in conversation_history:
53
68
  if not isinstance(message, Mapping):
54
69
  continue
55
- if _contains_routing_marker(message.get("api_content")):
70
+ if _contains_marker(message.get("api_content"), marker):
56
71
  return True
57
- if _contains_routing_marker(message.get("content")):
72
+ if _contains_marker(message.get("content"), marker):
58
73
  return True
59
74
  return False
60
75
 
@@ -62,9 +77,18 @@ def routing_guidance_present(conversation_history: object) -> bool:
62
77
  def routing_context_hook(**kwargs: Any) -> dict[str, str] | None:
63
78
  """Hermes ``pre_llm_call`` hook injecting versioned routing guidance once."""
64
79
 
80
+ history = kwargs.get("conversation_history")
65
81
  config = load_config()
66
- if not routing_context_enabled(config) or not tinyfish_mcp_configured(config):
82
+ if not routing_context_enabled(config):
83
+ return None
84
+ if not _api_key():
85
+ if not tinyfish_backend_configured(config):
86
+ return None
87
+ if routing_guidance_present(history, SETUP_CONTEXT_MARKER):
88
+ return None
89
+ return {"context": SETUP_GUIDANCE}
90
+ if not tinyfish_mcp_configured(config):
67
91
  return None
68
- if routing_guidance_present(kwargs.get("conversation_history")):
92
+ if routing_guidance_present(history):
69
93
  return None
70
94
  return {"context": ROUTING_GUIDANCE}
@@ -5,6 +5,9 @@ from __future__ import annotations
5
5
  import argparse
6
6
  import json
7
7
  import os
8
+ import re
9
+ import shutil
10
+ import subprocess
8
11
  import sys
9
12
  from typing import Any
10
13
 
@@ -195,6 +198,28 @@ def _prompt_secret(question: str) -> str:
195
198
  return ""
196
199
 
197
200
 
201
+ def _prompt_for_key() -> str:
202
+ print(f"TinyFish needs an API key. Create one (free) at {API_KEY_URL}")
203
+ if _confirm("Open that page in your browser?", default=True):
204
+ import webbrowser
205
+
206
+ webbrowser.open(API_KEY_URL)
207
+ return _prompt_secret("Paste your TinyFish API key (Enter to skip): ")
208
+
209
+
210
+ def _validate_key(api_key: str) -> str | None:
211
+ """Return an error message when TinyFish rejects the key; one free search."""
212
+
213
+ try:
214
+ rest_client.search("TinyFish", api_key=api_key, timeout=15.0)
215
+ except rest_client.TinyFishRestError as exc:
216
+ if exc.status in (401, 403):
217
+ return f"TinyFish rejected that API key ({exc}). Get one at {API_KEY_URL}"
218
+ # Network trouble shouldn't block saving a key that may be fine.
219
+ print(f"Warning: could not verify the key ({exc}); saving it anyway.")
220
+ return None
221
+
222
+
198
223
  def _apply_web_backend_config(config: dict[str, Any]) -> None:
199
224
  web = config.setdefault("web", {})
200
225
  web["search_backend"] = "tinyfish"
@@ -231,8 +256,25 @@ def cmd_setup(
231
256
  *,
232
257
  provider: TinyFishWebSearchProvider | None = None,
233
258
  ) -> int:
234
- config = _load_config()
259
+ api_key = (getattr(args, "api_key", None) or "").strip()
260
+ existing = _api_key_env_var()
261
+ if not api_key and existing:
262
+ print(f"Using the TinyFish API key from {existing}")
263
+ elif not api_key and sys.stdin.isatty():
264
+ api_key = _prompt_for_key()
265
+ if api_key:
266
+ rejection = _validate_key(api_key)
267
+ if rejection:
268
+ print(rejection, file=sys.stderr)
269
+ return 1
270
+ if not _store_api_key(api_key):
271
+ return 1
272
+ elif not existing:
273
+ print(MISSING_KEY_ERROR, file=sys.stderr)
274
+ return 1
235
275
 
276
+ # Route web tools only once a usable key exists, so a failed setup changes nothing.
277
+ config = _load_config()
236
278
  if not getattr(args, "no_web_backend", False) and _confirm(
237
279
  "Set Hermes web.search_backend and web.extract_backend to tinyfish?",
238
280
  default=True,
@@ -241,17 +283,8 @@ def cmd_setup(
241
283
  _apply_web_backend_config(config)
242
284
  _save_config(config)
243
285
  print("Configured Hermes web backends to use TinyFish")
244
-
245
- api_key = (getattr(args, "api_key", None) or "").strip()
246
- if (
247
- not api_key
248
- and not _api_key_env_var()
249
- and sys.stdin.isatty()
250
- and _confirm("Add TINYFISH_API_KEY now?", default=True, assume_yes=False)
251
- ):
252
- api_key = _prompt_secret(f"TinyFish API key (create at {API_KEY_URL}): ")
253
- if api_key and not _store_api_key(api_key):
254
- return 1
286
+ if not _enable_toolset("web"):
287
+ return 1
255
288
 
256
289
  if getattr(args, "live", False):
257
290
  doctor_args = argparse.Namespace(json=False, live=True, live_paid=False)
@@ -518,6 +551,69 @@ def _policy_effect_line(policy: str) -> str:
518
551
  return "Policy 'request': each TinyFish browser session asks for approval."
519
552
 
520
553
 
554
+ AGENT_BROWSER_SETUP = ("tools", "post-setup", "agent_browser")
555
+
556
+
557
+ def _hermes(*args: str, capture: bool = True) -> subprocess.CompletedProcess[str]:
558
+ # The running interpreter is Hermes'; a `hermes` on PATH may be another install.
559
+ return subprocess.run(
560
+ [sys.executable, "-m", "hermes_cli.main", *args],
561
+ capture_output=capture,
562
+ text=True,
563
+ check=False,
564
+ )
565
+
566
+
567
+ def _agent_browser_installed() -> bool:
568
+ try:
569
+ import pm
570
+
571
+ if pm.installed_package("agent-browser"):
572
+ return True
573
+ except Exception: # pm is Hermes-internal; fall back to PATH
574
+ pass
575
+ return shutil.which("agent-browser") is not None
576
+
577
+
578
+ def _toolset_disabled(name: str) -> bool | None:
579
+ result = _hermes("tools", "list")
580
+ if result.returncode != 0:
581
+ return None
582
+ return re.search(rf"\bdisabled\s+{re.escape(name)}\s", result.stdout) is not None
583
+
584
+
585
+ def _enable_toolset(name: str) -> bool:
586
+ # `tools enable` pins an explicit toolset list, so only touch a disabled toolset.
587
+ disabled = _toolset_disabled(name)
588
+ if disabled is False:
589
+ return True
590
+ result = _hermes("tools", "enable", name) if disabled else None
591
+ if result is not None and result.returncode == 0:
592
+ print(f"Enabled Hermes' {name} toolset (CLI); start a new session to use it.")
593
+ return True
594
+ print(
595
+ f"Could not enable the {name} toolset; run `hermes tools enable {name}`.",
596
+ file=sys.stderr,
597
+ )
598
+ return False
599
+
600
+
601
+ def _ensure_agent_browser() -> bool:
602
+ if _agent_browser_installed():
603
+ return True
604
+ command = " ".join(("hermes", *AGENT_BROWSER_SETUP))
605
+ print("Hermes' browser tools need the agent-browser CLI, which is not installed.")
606
+ if _confirm("Install it now?", default=True):
607
+ # post-setup exits 0 even when the install fails, so re-check instead.
608
+ _hermes(*AGENT_BROWSER_SETUP, capture=False)
609
+ if _agent_browser_installed():
610
+ return True
611
+ print(f"Install failed; retry with `{command}`.", file=sys.stderr)
612
+ return False
613
+ print(f"Install it with `{command}`.")
614
+ return True
615
+
616
+
521
617
  def cmd_browser(args: argparse.Namespace) -> int:
522
618
  subcommand = getattr(args, "browser_command", None) or "status"
523
619
  config = _load_config()
@@ -535,7 +631,8 @@ def cmd_browser(args: argparse.Namespace) -> int:
535
631
  _save_config(config)
536
632
  print("Set browser.cloud_provider to tinyfish.")
537
633
  print(_policy_effect_line(policy))
538
- return 0
634
+ toolset_ok = _enable_toolset("browser")
635
+ return 0 if _ensure_agent_browser() and toolset_ok else 1
539
636
  if subcommand == "disable":
540
637
  if current != "tinyfish":
541
638
  print("TinyFish is not Hermes' browser cloud provider; nothing to change.")