hermes-plugin-tinyfish 0.3.4__tar.gz → 0.4.0__tar.gz

{hermes_plugin_tinyfish-0.3.4 → hermes_plugin_tinyfish-0.4.0}/CHANGELOG.md

@@ -7,6 +7,43 @@ and this project uses semantic versioning.
 
 ## Unreleased
 
+## [0.4.0] - 2026-08-29
+
+### Added
+
+- Added current TinyFish Search filters (`include_domains`,
+  `exclude_domains`, `pub_year_min`, and `pub_year_max`) and Fetch controls
+  (`purpose`, CSS include/exclude selectors, conditional validators, and
+  validator-return metadata) across both MCP and REST transports.
+- Added compatibility with TinyFish CLI-seeded `MCP_TINYFISH_API_KEY`, while
+  preserving `TINYFISH_API_KEY` precedence.
+- Added a Hermes `after-install.md` guide that routes new installs through
+  OAuth-first setup, diagnostics, and explicit paid-Browser policy.
+- Added a minimal `hermes/` Git-install distribution so current Hermes security
+  scanning evaluates runtime plugin files instead of repository development
+  workflows and agent-maintenance tooling.
+
+### Changed
+
+- Forwarded operator Search/Fetch defaults consistently through MCP and REST,
+  increased the REST Fetch timeout to TinyFish's documented 150-second batch
+  recommendation, chunked direct calls at the 10-URL API cap, and restored
+  output to input URL order.
+- Expanded normalization to preserve news/research fields, selector recovery
+  hints, conditional/cache metadata, authorship, links/images, and latency;
+  JSON Fetch document trees now remain valid JSON.
+- Bumped the tool-routing context to version 3 so it distinguishes transport-
+  invariant operator defaults from one-off controls that require native MCP
+  schemas.
+
+### Fixed
+
+- Closed billable Browser sessions when TinyFish returns an ID without a CDP
+  URL, prevented cleanup exceptions from escaping the provider contract, and
+  withheld Browser registration on Hermes hosts lacking approval hooks.
+- Adopted Hermes's managed-secret credential writer with persisted-value
+  verification while retaining the legacy writer on older supported hosts.
+
 ## [0.3.4] - 2026-08-13
 
 ### Fixed

{hermes_plugin_tinyfish-0.3.4 → hermes_plugin_tinyfish-0.4.0}/CONTRIBUTING.md

@@ -11,7 +11,7 @@ python -m pip install -U pip
 python -m pip install -e ".[dev]"
 ruff format .
 ruff check .
-mypy hermes_plugin_tinyfish
+mypy hermes/hermes_plugin_tinyfish
 pytest
 python -m build
 ```

{hermes_plugin_tinyfish-0.3.4 → hermes_plugin_tinyfish-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: hermes-plugin-tinyfish
-Version: 0.3.4
+Version: 0.4.0
 Summary: TinyFish Search, Fetch, plain-language routing, update awareness, and optional Browser for Hermes Agent
 Project-URL: Homepage, https://github.com/gabeosx/hermes-plugin-tinyfish
 Project-URL: Documentation, https://github.com/gabeosx/hermes-plugin-tinyfish#readme
@@ -59,7 +59,7 @@ TinyFish documentation and may change; Browser defaults to `deny`.
 Hermes Git plugin install:
 
 ```bash
-hermes plugins install gabeosx/hermes-plugin-tinyfish --enable
+hermes plugins install gabeosx/hermes-plugin-tinyfish/hermes --enable
 hermes tinyfish setup
 ```
 
@@ -81,6 +81,16 @@ hermes tinyfish status
 hermes tinyfish doctor
 ```
 
+Installations created before `0.4.0` from the repository root should migrate
+once to the minimal scanner-friendly distribution before updating:
+
+```bash
+hermes plugins install gabeosx/hermes-plugin-tinyfish/hermes --force --enable
+```
+
+This replaces only the plugin checkout; Hermes configuration, OAuth state, and
+externally managed credentials remain in the active Hermes profile.
+
 If the plugin was installed as a Python package instead, update it with:
 
 ```bash
@@ -130,8 +140,10 @@ tinyfish:
 ```
 
 The plugin prefers TinyFish's hosted OAuth MCP server and falls back to REST
-with `TINYFISH_API_KEY` when MCP OAuth is unavailable. It may save
-`TINYFISH_API_KEY` to `~/.hermes/.env` if you choose API-key fallback.
+with `TINYFISH_API_KEY` when MCP OAuth is unavailable. It also recognizes
+`MCP_TINYFISH_API_KEY`, which the TinyFish CLI may seed during its Hermes
+connection flow; an explicit `TINYFISH_API_KEY` takes precedence. The plugin
+may save `TINYFISH_API_KEY` to `~/.hermes/.env` if you choose API-key fallback.
 
 Verify Search and Fetch:
 
@@ -214,28 +226,70 @@ compressed.
 
 ## Search and Fetch Options
 
-Optional REST fallback defaults can be configured in `config.yaml`:
+Optional operator defaults can be configured in `config.yaml`:
 
 ```yaml
 tinyfish:
   search:
     location: US
     language: en
+    include_domains: docs.tinyfish.ai,github.com
+    exclude_domains: example.com
     recency_minutes: 1440
     domain_type: web
+    # pub_year_min: 2024  # research_paper mode only
+    # pub_year_max: 2026
     page: 0
+    purpose: Find primary documentation for implementation decisions
   fetch:
     format: markdown
     links: false
     image_links: false
     ttl: 3600
+    per_url_timeout_ms: 45000
+    purpose: Extract the main documentation content
+    include_selectors: [main, article]
+    exclude_selectors: [nav, .comments, .newsletter-signup]
+    include_etag_and_last_modified: true
 ```
 
-MCP remains the preferred path when configured. These options apply only to
-REST fallback calls. They are fixed fallback defaults, not required settings
-for ordinary requests and not a substitute for plain-language, per-request MCP
-controls. The plugin does not add persistent include-domain, exclude-selector,
-or similar parity settings.
+MCP remains the preferred path when configured. These defaults are forwarded
+consistently through MCP and REST, so failover does not silently discard the
+operator's domain, date, intent, selector, cache, or extraction constraints.
+They are not required for ordinary requests and do not replace plain-language,
+per-request MCP controls: Hermes's generic `web_search` and `web_extract`
+schemas cannot express these options for one call, so the routing note steers
+one-off advanced requests to native TinyFish `search` or `fetch_content`.
+
+Fetch also supports the URL-specific `if_none_match` and
+`if_modified_since` settings. TinyFish permits either validator only for a
+single URL, and the plugin fails clearly instead of sending an invalid batch.
+Selector lists accept up to 20 CSS selectors. A total include-selector miss is
+returned as `selector_not_matched` with TinyFish's unmatched selectors and
+candidate-selector retry hints preserved in document metadata. Conditional
+validators, `not_modified`, links, images, author/date, latency, output format,
+and other current Fetch metadata are preserved as well. JSON-format document
+trees are serialized as valid JSON rather than Python object representations.
+
+Direct provider callers may pass the same Fetch options per call. Requests over
+TinyFish's 10-URL API limit are split into bounded chunks, and results are
+restored to input URL order even when TinyFish separates successes and errors.
+
+## Why Choose This Community Plugin
+
+- OAuth-first MCP setup avoids requiring an API key, while config-aware REST
+  fallback and TinyFish CLI-seeded keys remain supported.
+- Search and Fetch use the same advanced option set on both transports, with
+  categorized MCP health, explicit MCP-only/REST-only diagnostics, safe
+  reauthorization, and observable failover.
+- Browser is fail-closed by default. It is not exposed on older Hermes hosts
+  that cannot install the approval hook, incomplete billable sessions are
+  cleaned up, and close failures never escape Hermes's cleanup contract.
+- GitHub and PyPI distribution, nonblocking update awareness, migration
+  reporting, and an automated Hermes compatibility matrix make upgrades
+  predictable.
+- The scope stays Hermes-native: no competing delegated agent loop, no hidden
+  expansion of paid capabilities, and clear independent/pricing disclosures.
 
 ## Optional Browser Provider
 
@@ -344,7 +398,7 @@ python -m pip install -U pip
 python -m pip install -e ".[dev]"
 ruff format .
 ruff check .
-mypy hermes_plugin_tinyfish
+mypy hermes/hermes_plugin_tinyfish
 pytest --cov=hermes_plugin_tinyfish --cov-fail-under=70
 python -m build
 ```

{hermes_plugin_tinyfish-0.3.4 → hermes_plugin_tinyfish-0.4.0}/README.md

@@ -27,7 +27,7 @@ TinyFish documentation and may change; Browser defaults to `deny`.
 Hermes Git plugin install:
 
 ```bash
-hermes plugins install gabeosx/hermes-plugin-tinyfish --enable
+hermes plugins install gabeosx/hermes-plugin-tinyfish/hermes --enable
 hermes tinyfish setup
 ```
 
@@ -49,6 +49,16 @@ hermes tinyfish status
 hermes tinyfish doctor
 ```
 
+Installations created before `0.4.0` from the repository root should migrate
+once to the minimal scanner-friendly distribution before updating:
+
+```bash
+hermes plugins install gabeosx/hermes-plugin-tinyfish/hermes --force --enable
+```
+
+This replaces only the plugin checkout; Hermes configuration, OAuth state, and
+externally managed credentials remain in the active Hermes profile.
+
 If the plugin was installed as a Python package instead, update it with:
 
 ```bash
@@ -98,8 +108,10 @@ tinyfish:
 ```
 
 The plugin prefers TinyFish's hosted OAuth MCP server and falls back to REST
-with `TINYFISH_API_KEY` when MCP OAuth is unavailable. It may save
-`TINYFISH_API_KEY` to `~/.hermes/.env` if you choose API-key fallback.
+with `TINYFISH_API_KEY` when MCP OAuth is unavailable. It also recognizes
+`MCP_TINYFISH_API_KEY`, which the TinyFish CLI may seed during its Hermes
+connection flow; an explicit `TINYFISH_API_KEY` takes precedence. The plugin
+may save `TINYFISH_API_KEY` to `~/.hermes/.env` if you choose API-key fallback.
 
 Verify Search and Fetch:
 
@@ -182,28 +194,70 @@ compressed.
 
 ## Search and Fetch Options
 
-Optional REST fallback defaults can be configured in `config.yaml`:
+Optional operator defaults can be configured in `config.yaml`:
 
 ```yaml
 tinyfish:
   search:
     location: US
     language: en
+    include_domains: docs.tinyfish.ai,github.com
+    exclude_domains: example.com
     recency_minutes: 1440
     domain_type: web
+    # pub_year_min: 2024  # research_paper mode only
+    # pub_year_max: 2026
     page: 0
+    purpose: Find primary documentation for implementation decisions
   fetch:
     format: markdown
     links: false
     image_links: false
     ttl: 3600
+    per_url_timeout_ms: 45000
+    purpose: Extract the main documentation content
+    include_selectors: [main, article]
+    exclude_selectors: [nav, .comments, .newsletter-signup]
+    include_etag_and_last_modified: true
 ```
 
-MCP remains the preferred path when configured. These options apply only to
-REST fallback calls. They are fixed fallback defaults, not required settings
-for ordinary requests and not a substitute for plain-language, per-request MCP
-controls. The plugin does not add persistent include-domain, exclude-selector,
-or similar parity settings.
+MCP remains the preferred path when configured. These defaults are forwarded
+consistently through MCP and REST, so failover does not silently discard the
+operator's domain, date, intent, selector, cache, or extraction constraints.
+They are not required for ordinary requests and do not replace plain-language,
+per-request MCP controls: Hermes's generic `web_search` and `web_extract`
+schemas cannot express these options for one call, so the routing note steers
+one-off advanced requests to native TinyFish `search` or `fetch_content`.
+
+Fetch also supports the URL-specific `if_none_match` and
+`if_modified_since` settings. TinyFish permits either validator only for a
+single URL, and the plugin fails clearly instead of sending an invalid batch.
+Selector lists accept up to 20 CSS selectors. A total include-selector miss is
+returned as `selector_not_matched` with TinyFish's unmatched selectors and
+candidate-selector retry hints preserved in document metadata. Conditional
+validators, `not_modified`, links, images, author/date, latency, output format,
+and other current Fetch metadata are preserved as well. JSON-format document
+trees are serialized as valid JSON rather than Python object representations.
+
+Direct provider callers may pass the same Fetch options per call. Requests over
+TinyFish's 10-URL API limit are split into bounded chunks, and results are
+restored to input URL order even when TinyFish separates successes and errors.
+
+## Why Choose This Community Plugin
+
+- OAuth-first MCP setup avoids requiring an API key, while config-aware REST
+  fallback and TinyFish CLI-seeded keys remain supported.
+- Search and Fetch use the same advanced option set on both transports, with
+  categorized MCP health, explicit MCP-only/REST-only diagnostics, safe
+  reauthorization, and observable failover.
+- Browser is fail-closed by default. It is not exposed on older Hermes hosts
+  that cannot install the approval hook, incomplete billable sessions are
+  cleaned up, and close failures never escape Hermes's cleanup contract.
+- GitHub and PyPI distribution, nonblocking update awareness, migration
+  reporting, and an automated Hermes compatibility matrix make upgrades
+  predictable.
+- The scope stays Hermes-native: no competing delegated agent loop, no hidden
+  expansion of paid capabilities, and clear independent/pricing disclosures.
 
 ## Optional Browser Provider
 
@@ -312,7 +366,7 @@ python -m pip install -U pip
 python -m pip install -e ".[dev]"
 ruff format .
 ruff check .
-mypy hermes_plugin_tinyfish
+mypy hermes/hermes_plugin_tinyfish
 pytest --cov=hermes_plugin_tinyfish --cov-fail-under=70
 python -m build
 ```

hermes_plugin_tinyfish-0.4.0/__init__.py

@@ -0,0 +1,19 @@
+"""Compatibility entry point for pre-0.4 root-directory Git installs.
+
+New Hermes installations use the scanner-friendly ``hermes/`` subdirectory.
+Existing root-directory installations still load this shim after an update.
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+_INSTALL_ROOT = Path(__file__).resolve().parent / "hermes"
+_install_root_text = str(_INSTALL_ROOT)
+if _install_root_text not in sys.path:
+    sys.path.insert(0, _install_root_text)
+
+from hermes_plugin_tinyfish import register  # noqa: E402
+
+__all__ = ["register"]

hermes_plugin_tinyfish-0.4.0/after-install.md

@@ -0,0 +1,12 @@
+# TinyFish for Hermes has moved to a minimal install surface
+
+This compatibility entry point keeps existing installations working. For
+future scanner-clean updates, reinstall from the dedicated subdirectory:
+
+```bash
+hermes plugins install gabeosx/hermes-plugin-tinyfish/hermes --force --enable
+hermes tinyfish setup
+```
+
+The reinstall preserves your Hermes configuration, OAuth state, and externally
+managed API keys.

hermes_plugin_tinyfish-0.4.0/hermes/after-install.md

@@ -0,0 +1,21 @@
+# TinyFish for Hermes is installed
+
+Finish setup with:
+
+```bash
+hermes tinyfish setup
+```
+
+The setup flow prefers TinyFish OAuth, can use `TINYFISH_API_KEY` or the
+TinyFish CLI's `MCP_TINYFISH_API_KEY`, and verifies the active transport.
+
+Then check the installation with:
+
+```bash
+hermes tinyfish doctor
+```
+
+Search and Fetch are free and enabled through Hermes. Paid Browser sessions
+remain disabled until you explicitly choose an approval policy during setup.
+
+Documentation: https://github.com/gabeosx/hermes-plugin-tinyfish#readme

{hermes_plugin_tinyfish-0.3.4 → hermes_plugin_tinyfish-0.4.0/hermes}/hermes_plugin_tinyfish/__init__.py

@@ -52,9 +52,11 @@ def register(ctx: Any) -> None:
     update_checker = UpdateChecker(resolve_install_info(ctx, __version__))
     turn_context = TinyFishTurnContext(update_checker)
     ctx.register_web_search_provider(provider)
-    if hasattr(ctx, "register_browser_provider"):
-        ctx.register_browser_provider(TinyFishBrowserProvider())
     if hasattr(ctx, "register_hook"):
+        # Browser policy "request" depends on the pre_tool_call approval hook.
+        # Do not expose the paid provider on older hosts that cannot install it.
+        if hasattr(ctx, "register_browser_provider"):
+            ctx.register_browser_provider(TinyFishBrowserProvider())
         ctx.register_hook("pre_tool_call", pre_tool_call_policy)
         ctx.register_hook("pre_llm_call", turn_context)
     update_checker.start()

{hermes_plugin_tinyfish-0.3.4 → hermes_plugin_tinyfish-0.4.0/hermes}/hermes_plugin_tinyfish/browser_provider.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 import logging
 import uuid
+from contextlib import suppress
 from typing import Any
 
 try:
@@ -17,7 +18,7 @@ except Exception:  # pragma: no cover - lets package import outside Hermes
 from . import rest_client
 from .config import credit_policy, tinyfish_config
 from .credit_policy import INDEPENDENT_NOTICE, PRICING_NOTICE
-from .provider import _provider_env
+from .provider import _api_key
 
 logger = logging.getLogger(__name__)
 
@@ -34,12 +35,14 @@ class TinyFishBrowserProvider(_HermesBrowserProvider):  # type: ignore[misc]
         return "TinyFish"
 
     def is_available(self) -> bool:
-        return bool(credit_policy("browser") in {"request", "allow"} and _provider_env("TINYFISH_API_KEY"))
+        return bool(credit_policy("browser") in {"request", "allow"} and _api_key())
 
     def create_session(self, task_id: str) -> dict[str, object]:
-        api_key = _provider_env("TINYFISH_API_KEY")
+        api_key = _api_key()
         if not api_key:
-            raise ValueError("TINYFISH_API_KEY is required for TinyFish Browser sessions.")
+            raise ValueError(
+                "TINYFISH_API_KEY or MCP_TINYFISH_API_KEY is required for TinyFish Browser sessions."
+            )
         if credit_policy("browser") == "deny":
             raise ValueError(
                 "TinyFish Browser is credit-policy denied. Run "
@@ -51,12 +54,18 @@ class TinyFishBrowserProvider(_HermesBrowserProvider):  # type: ignore[misc]
             try:
                 timeout_seconds = int(browser_cfg["timeout_seconds"])
             except (TypeError, ValueError):
+                logger.warning("Ignoring malformed TinyFish Browser timeout configuration")
                 timeout_seconds = None
 
         data = rest_client.create_browser_session(api_key=api_key, timeout_seconds=timeout_seconds)
         session_id = str(data.get("session_id") or data.get("id") or "")
         cdp_url = str(data.get("cdp_url") or data.get("cdpUrl") or "")
         if not session_id or not cdp_url:
+            if session_id:
+                # Avoid leaving a billable half-created session running when
+                # TinyFish returns an incomplete connection payload.
+                with suppress(Exception):
+                    rest_client.close_browser_session(session_id, api_key=api_key)
             raise RuntimeError("TinyFish Browser did not return session_id and cdp_url.")
         session_name = f"tinyfish_{task_id}_{uuid.uuid4().hex[:8]}"
         logger.info("Created TinyFish browser session")
@@ -72,11 +81,16 @@ class TinyFishBrowserProvider(_HermesBrowserProvider):  # type: ignore[misc]
         }
 
     def close_session(self, session_id: str) -> bool:
-        api_key = _provider_env("TINYFISH_API_KEY")
-        if not api_key:
-            logger.warning("Cannot close TinyFish browser session: missing API key")
+        # Hermes' BrowserProvider contract treats cleanup as best effort.
+        try:
+            api_key = _api_key()
+            if not api_key:
+                logger.warning("Cannot close TinyFish browser session: missing API key")
+                return False
+            return rest_client.close_browser_session(session_id, api_key=api_key)
+        except Exception as exc:
+            logger.debug("TinyFish browser close failed (%s)", type(exc).__name__)
             return False
-        return rest_client.close_browser_session(session_id, api_key=api_key)
 
     def emergency_cleanup(self, session_id: str) -> None:
         try:

{hermes_plugin_tinyfish-0.3.4 → hermes_plugin_tinyfish-0.4.0/hermes}/hermes_plugin_tinyfish/config.py

@@ -135,16 +135,34 @@ def _bool_option(value: Any) -> bool | None:
     return str(value).strip().lower() in {"1", "true", "yes", "on"}
 
 
+def _string_list_option(value: Any) -> list[str] | None:
+    if value in (None, ""):
+        return None
+    if not isinstance(value, (list, tuple)):
+        return None
+    items = [str(item).strip() for item in value if str(item).strip()]
+    return items or None
+
+
 def search_options(config: dict[str, Any] | None = None) -> SearchOptions:
     section = tinyfish_config(config).get("search") or {}
     if not isinstance(section, dict):
         return {}
     options: SearchOptions = {}
-    for key in ("location", "language", "after_date", "before_date", "domain_type", "purpose"):
+    for key in (
+        "location",
+        "language",
+        "include_domains",
+        "exclude_domains",
+        "after_date",
+        "before_date",
+        "domain_type",
+        "purpose",
+    ):
         value = section.get(key)
         if value not in (None, ""):
             options[key] = str(value)
-    for key in ("recency_minutes", "page"):
+    for key in ("recency_minutes", "pub_year_min", "pub_year_max", "page"):
         value = _int_option(section.get(key))
         if value is not None:
             options[key] = value
@@ -157,13 +175,21 @@ def fetch_options(config: dict[str, Any] | None = None) -> FetchOptions:
         return {}
     options: FetchOptions = {}
     for key in ("ttl", "per_url_timeout_ms"):
-        value = _int_option(section.get(key))
-        if value is not None:
-            options[key] = value
-    for key in ("links", "image_links"):
-        value = _bool_option(section.get(key))
-        if value is not None:
-            options[key] = value
+        int_value = _int_option(section.get(key))
+        if int_value is not None:
+            options[key] = int_value
+    for key in ("links", "image_links", "include_etag_and_last_modified"):
+        bool_value = _bool_option(section.get(key))
+        if bool_value is not None:
+            options[key] = bool_value
+    for key in ("purpose", "if_none_match", "if_modified_since"):
+        text_value = section.get(key)
+        if text_value not in (None, ""):
+            options[key] = str(text_value)
+    for key in ("include_selectors", "exclude_selectors"):
+        list_value = _string_list_option(section.get(key))
+        if list_value is not None:
+            options[key] = list_value
     return options