modis-py-tools 0.1.0__tar.gz

modis_py_tools-0.1.0/PKG-INFO

@@ -0,0 +1,281 @@
+Metadata-Version: 2.4
+Name: modis-py-tools
+Version: 0.1.0
+Summary: Common MoDIS-compatible Python tools and tool-call execution helpers
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: pydantic>=2.7.0
+Provides-Extra: web
+Requires-Dist: valyu>=2.0.0; extra == "web"
+Provides-Extra: browser
+Requires-Dist: markdownify>=0.12.0; extra == "browser"
+Requires-Dist: playwright>=1.45.0; extra == "browser"
+Requires-Dist: readability-lxml>=0.8.1; extra == "browser"
+Provides-Extra: dev
+Requires-Dist: build>=1.2.0; extra == "dev"
+Requires-Dist: mypy>=1.10.0; extra == "dev"
+Requires-Dist: pytest>=8.0.0; extra == "dev"
+Requires-Dist: ruff>=0.8.0; extra == "dev"
+
+# MoDIS Python Tools
+
+`modis-py-tools` provides MoDIS-compatible function tool definitions, tool
+groups, and tool-call execution helpers for Python applications.
+
+The distribution name is currently `modis-py-tools`; the import package is
+`modis_tools`.
+
+## Install
+
+```bash
+python -m pip install modis-py-tools
+```
+
+Optional web provider dependency:
+
+```bash
+python -m pip install "modis-py-tools[web]"
+```
+
+Development:
+
+```bash
+python -m pip install -e ".[dev,web]"
+.venv/bin/python -m pytest
+.venv/bin/python -m ruff check .
+.venv/bin/python -m ruff format --check .
+.venv/bin/python -m mypy src
+.venv/bin/python -m build
+```
+
+## Function Conversion
+
+```python
+from modis_tools import function_to_tool
+
+def lookup_order(order_id: str, include_history: bool = False) -> dict:
+    """Look up an order.
+
+    Args:
+        order_id: Order identifier.
+        include_history: Whether to include status history.
+    """
+    return {"order_id": order_id, "status": "processing"}
+
+tool = function_to_tool(lookup_order, name="orders.lookup")
+tool_definition = tool.to_wire()
+```
+
+## Registry Execution
+
+```python
+from modis_tools import ToolRegistry
+from modis_tools.builtins import python_group, shell_group
+
+registry = ToolRegistry()
+registry.include(shell_group())
+registry.include(python_group())
+
+tool_definitions = registry.definitions()
+results = registry.execute_tool_calls(model_tool_calls)
+messages = [result.to_chat_message() for result in results]
+```
+
+`execute_tool_call()` and `execute_tool_calls()` return failed `ToolResult`
+objects by default for unknown tools, invalid arguments, and tool exceptions.
+Pass `raise_on_error=True` when the host application should handle exceptions.
+
+For a fuller host integration guide, including unattended pipeline setup and
+future image generation extension notes, see `docs/INTEGRATION.md`.
+
+## Built-In Tools
+
+```python
+from modis_tools.builtins import python_group, shell_group, skills_group, web_group
+
+shell_tools = shell_group()
+python_tools = python_group()
+skill_tools = skills_group(skill_home="./skills")
+web_tools = web_group(api_key="...")
+```
+
+`shell.run` captures `stdout`, `stderr`, exit code, duration, timeout state,
+working directory, policy decision metadata, and truncation metadata.
+`python.run` executes Python source in a subprocess using the current
+interpreter by default.
+
+Shell and Python tools execute local code. Only expose them in trusted contexts
+where the caller owns policy, sandboxing, and approval decisions.
+
+Shell tools support host-owned policies. Policies are bound by the application
+when registering the group and are not exposed as model tool-call parameters.
+
+```python
+from modis_tools import ToolRegistry
+from modis_tools.builtins import ShellPolicy, shell_group
+
+registry = ToolRegistry()
+registry.include(shell_group(policy=ShellPolicy.readonly_project("./")))
+```
+
+Built-in shell policy profiles:
+
+| Profile | Purpose |
+| --- | --- |
+| `trusted_local` | Backward-compatible default for trusted local use. |
+| `disabled` | Denies every shell request. |
+| `restricted` | Deterministic allowlist for unattended pipelines. |
+| `readonly_project` | Read-oriented `argv` commands inside one project root. |
+
+`shell.run` accepts exactly one of `cmd` or `argv`. `cmd` executes with
+`shell=True`; `argv` executes with `shell=False`. Restricted policies should
+prefer `argv` mode. Policy controls are guardrails, not a strong OS sandbox; use
+containers, VMs, or other isolation for untrusted execution.
+
+## Skill Tools
+
+Skill support is split into runtime visibility, optional host-owned matching, and
+tool-based skill use. `modis-tools` provides the use layer: a registry, a prompt
+generator, and `skills.*` tools for progressive loading.
+
+`skill_home` is required in every mode because all skill files must resolve from
+a trusted root.
+
+```python
+from modis_tools import ToolRegistry
+from modis_tools.builtins import skills_group
+from modis_tools.skills import SkillRegistry, build_skill_system_prompt
+
+skill_home = "./skills"
+registry = ToolRegistry()
+registry.include(skills_group(skill_home=skill_home, mode="hybrid"))
+
+skill_registry = SkillRegistry.from_home(skill_home)
+system_prompt = build_skill_system_prompt(skill_registry, mode="hybrid")
+```
+
+The skills group exposes:
+
+- `skills.list()`
+- `skills.read(name)`
+- `skills.list_resources(name)`
+- `skills.read_resource(name, path)`
+- `skills.system_prompt(active_skill=None)`
+
+Execution modes:
+
+| Mode | Behavior |
+| --- | --- |
+| `tools_only` | Models use only `skills.*` tools to read instructions and resources. |
+| `shell_only` | Prompt tells the model where `skill_home` is; host shell/file policy must handle access. |
+| `hybrid` | Models use `skills.*` for reads and policy-gated shell for commands only when needed. |
+
+Skill matching is intentionally optional and host-owned. The package defines
+`PromptSkillEvaluator` as a protocol, but does not decide when a skill should be
+used.
+
+## Web Tools
+
+The web group exposes:
+
+- `web.search(query, num_results=10, search_type="all", relevance_threshold=0.5, included_sources=None, excluded_sources=None, country_code=None, response_length=None, category=None, start_date=None, end_date=None, max_price=None, fast_mode=False, url_only=False, source_biases=None, instructions=None)`
+- `web.open(id=None, cursor=-1, loc=-1, num_lines=-1)`
+- `web.find(pattern, cursor=-1, context_lines=3)`
+
+Valyu is the default provider when no provider is supplied. Search uses Valyu
+search, and direct URL opens use Valyu contents extraction.
+Opened pages and find results include source ids, URLs, and one-based line
+ranges so responses can be cited or re-opened later in the same tool session.
+
+```python
+from modis_tools import ToolRegistry
+from modis_tools.builtins import web_group
+
+registry = ToolRegistry()
+registry.include(web_group(api_key="..."))
+```
+
+Search options map to Valyu search controls. For example:
+
+```python
+result = registry.execute_tool_call({
+    "id": "search_1",
+    "type": "function",
+    "function": {
+        "name": "web.search",
+        "arguments": """
+        {
+          "query": "recent multimodal retrieval papers",
+          "num_results": 5,
+          "relevance_threshold": 0.45,
+          "included_sources": ["valyu/valyu-arxiv"],
+          "response_length": "medium",
+          "start_date": "2026-04-29",
+          "end_date": "2026-05-06",
+          "country_code": "US"
+        }
+        """
+    }
+})
+```
+
+Default search values are copied from the installed Valyu SDK where exposed:
+
+| Option | Default | Notes |
+| --- | --- | --- |
+| `num_results` | `10` | Sent to Valyu as `max_num_results`. |
+| `search_type` | `"all"` | Valyu scopes are `web`, `proprietary`, `all`, and `news`. |
+| `relevance_threshold` | `0.5` | Set to `None` to omit the threshold parameter. |
+| `included_sources` | `None` | Valyu source ids or arbitrary URLs. |
+| `excluded_sources` | `None` | Valyu source ids or arbitrary URLs. |
+| `country_code` | `None` | Optional ISO country code. |
+| `response_length` | `None` | Supports `short`, `medium`, `large`, `max`, integer, or numeric string. |
+| `category` | `None` | Provider-specific category. |
+| `start_date` / `end_date` | `None` | Optional `YYYY-MM-DD` bounds. |
+| `max_price` | `None` | Provider cost limit if used. |
+| `fast_mode` | `False` | Sent explicitly. |
+| `url_only` | `False` | Sent explicitly. |
+| `source_biases` | `None` | Optional per-source integer biases. |
+| `instructions` | `None` | Optional provider instructions. |
+
+`is_tool_call` is not exposed to the model; the Valyu provider sends it as
+`True`.
+
+Valyu provider metadata that is not part of the normalized result shape, such as
+scores or ranking fields when returned by the SDK, is preserved in each result's
+`metadata` object.
+
+Custom providers implement `SearchProvider`:
+
+```python
+from modis_tools.providers.web import Page, SearchResult
+
+class MyProvider:
+    def search(self, query: str, *, num_results: int = 10, **kwargs: object) -> list[SearchResult]:
+        return [SearchResult(title="Example", url="https://example.test", content="Page text")]
+
+    def fetch(self, url: str) -> Page:
+        return Page(title="Fetched", url=url, markdown="Fetched markdown")
+```
+
+`web.open` can open prior search results by index/result id/URL, or an arbitrary
+direct URL. Browser-backed fetching is still kept behind the optional `browser`
+extra for future extension.
+
+## Live Web Tests
+
+Live web tests are skipped unless `VALYU_API_KEY` is set and the `web` extra is
+installed:
+
+```bash
+python -m pip install -e ".[dev,web]"
+VALYU_API_KEY=... pytest -m live
+```
+
+The broader web quality eval is opt-in because it consumes live provider quota:
+
+```bash
+VALYU_API_KEY=... MODIS_TOOLS_RUN_WEB_EVAL=1 pytest -m "live and eval" -vv
+```
+
+See `docs/WEB_EVAL.md` for the current query set and comparison checklist.

modis_py_tools-0.1.0/README.md

@@ -0,0 +1,262 @@
+# MoDIS Python Tools
+
+`modis-py-tools` provides MoDIS-compatible function tool definitions, tool
+groups, and tool-call execution helpers for Python applications.
+
+The distribution name is currently `modis-py-tools`; the import package is
+`modis_tools`.
+
+## Install
+
+```bash
+python -m pip install modis-py-tools
+```
+
+Optional web provider dependency:
+
+```bash
+python -m pip install "modis-py-tools[web]"
+```
+
+Development:
+
+```bash
+python -m pip install -e ".[dev,web]"
+.venv/bin/python -m pytest
+.venv/bin/python -m ruff check .
+.venv/bin/python -m ruff format --check .
+.venv/bin/python -m mypy src
+.venv/bin/python -m build
+```
+
+## Function Conversion
+
+```python
+from modis_tools import function_to_tool
+
+def lookup_order(order_id: str, include_history: bool = False) -> dict:
+    """Look up an order.
+
+    Args:
+        order_id: Order identifier.
+        include_history: Whether to include status history.
+    """
+    return {"order_id": order_id, "status": "processing"}
+
+tool = function_to_tool(lookup_order, name="orders.lookup")
+tool_definition = tool.to_wire()
+```
+
+## Registry Execution
+
+```python
+from modis_tools import ToolRegistry
+from modis_tools.builtins import python_group, shell_group
+
+registry = ToolRegistry()
+registry.include(shell_group())
+registry.include(python_group())
+
+tool_definitions = registry.definitions()
+results = registry.execute_tool_calls(model_tool_calls)
+messages = [result.to_chat_message() for result in results]
+```
+
+`execute_tool_call()` and `execute_tool_calls()` return failed `ToolResult`
+objects by default for unknown tools, invalid arguments, and tool exceptions.
+Pass `raise_on_error=True` when the host application should handle exceptions.
+
+For a fuller host integration guide, including unattended pipeline setup and
+future image generation extension notes, see `docs/INTEGRATION.md`.
+
+## Built-In Tools
+
+```python
+from modis_tools.builtins import python_group, shell_group, skills_group, web_group
+
+shell_tools = shell_group()
+python_tools = python_group()
+skill_tools = skills_group(skill_home="./skills")
+web_tools = web_group(api_key="...")
+```
+
+`shell.run` captures `stdout`, `stderr`, exit code, duration, timeout state,
+working directory, policy decision metadata, and truncation metadata.
+`python.run` executes Python source in a subprocess using the current
+interpreter by default.
+
+Shell and Python tools execute local code. Only expose them in trusted contexts
+where the caller owns policy, sandboxing, and approval decisions.
+
+Shell tools support host-owned policies. Policies are bound by the application
+when registering the group and are not exposed as model tool-call parameters.
+
+```python
+from modis_tools import ToolRegistry
+from modis_tools.builtins import ShellPolicy, shell_group
+
+registry = ToolRegistry()
+registry.include(shell_group(policy=ShellPolicy.readonly_project("./")))
+```
+
+Built-in shell policy profiles:
+
+| Profile | Purpose |
+| --- | --- |
+| `trusted_local` | Backward-compatible default for trusted local use. |
+| `disabled` | Denies every shell request. |
+| `restricted` | Deterministic allowlist for unattended pipelines. |
+| `readonly_project` | Read-oriented `argv` commands inside one project root. |
+
+`shell.run` accepts exactly one of `cmd` or `argv`. `cmd` executes with
+`shell=True`; `argv` executes with `shell=False`. Restricted policies should
+prefer `argv` mode. Policy controls are guardrails, not a strong OS sandbox; use
+containers, VMs, or other isolation for untrusted execution.
+
+## Skill Tools
+
+Skill support is split into runtime visibility, optional host-owned matching, and
+tool-based skill use. `modis-tools` provides the use layer: a registry, a prompt
+generator, and `skills.*` tools for progressive loading.
+
+`skill_home` is required in every mode because all skill files must resolve from
+a trusted root.
+
+```python
+from modis_tools import ToolRegistry
+from modis_tools.builtins import skills_group
+from modis_tools.skills import SkillRegistry, build_skill_system_prompt
+
+skill_home = "./skills"
+registry = ToolRegistry()
+registry.include(skills_group(skill_home=skill_home, mode="hybrid"))
+
+skill_registry = SkillRegistry.from_home(skill_home)
+system_prompt = build_skill_system_prompt(skill_registry, mode="hybrid")
+```
+
+The skills group exposes:
+
+- `skills.list()`
+- `skills.read(name)`
+- `skills.list_resources(name)`
+- `skills.read_resource(name, path)`
+- `skills.system_prompt(active_skill=None)`
+
+Execution modes:
+
+| Mode | Behavior |
+| --- | --- |
+| `tools_only` | Models use only `skills.*` tools to read instructions and resources. |
+| `shell_only` | Prompt tells the model where `skill_home` is; host shell/file policy must handle access. |
+| `hybrid` | Models use `skills.*` for reads and policy-gated shell for commands only when needed. |
+
+Skill matching is intentionally optional and host-owned. The package defines
+`PromptSkillEvaluator` as a protocol, but does not decide when a skill should be
+used.
+
+## Web Tools
+
+The web group exposes:
+
+- `web.search(query, num_results=10, search_type="all", relevance_threshold=0.5, included_sources=None, excluded_sources=None, country_code=None, response_length=None, category=None, start_date=None, end_date=None, max_price=None, fast_mode=False, url_only=False, source_biases=None, instructions=None)`
+- `web.open(id=None, cursor=-1, loc=-1, num_lines=-1)`
+- `web.find(pattern, cursor=-1, context_lines=3)`
+
+Valyu is the default provider when no provider is supplied. Search uses Valyu
+search, and direct URL opens use Valyu contents extraction.
+Opened pages and find results include source ids, URLs, and one-based line
+ranges so responses can be cited or re-opened later in the same tool session.
+
+```python
+from modis_tools import ToolRegistry
+from modis_tools.builtins import web_group
+
+registry = ToolRegistry()
+registry.include(web_group(api_key="..."))
+```
+
+Search options map to Valyu search controls. For example:
+
+```python
+result = registry.execute_tool_call({
+    "id": "search_1",
+    "type": "function",
+    "function": {
+        "name": "web.search",
+        "arguments": """
+        {
+          "query": "recent multimodal retrieval papers",
+          "num_results": 5,
+          "relevance_threshold": 0.45,
+          "included_sources": ["valyu/valyu-arxiv"],
+          "response_length": "medium",
+          "start_date": "2026-04-29",
+          "end_date": "2026-05-06",
+          "country_code": "US"
+        }
+        """
+    }
+})
+```
+
+Default search values are copied from the installed Valyu SDK where exposed:
+
+| Option | Default | Notes |
+| --- | --- | --- |
+| `num_results` | `10` | Sent to Valyu as `max_num_results`. |
+| `search_type` | `"all"` | Valyu scopes are `web`, `proprietary`, `all`, and `news`. |
+| `relevance_threshold` | `0.5` | Set to `None` to omit the threshold parameter. |
+| `included_sources` | `None` | Valyu source ids or arbitrary URLs. |
+| `excluded_sources` | `None` | Valyu source ids or arbitrary URLs. |
+| `country_code` | `None` | Optional ISO country code. |
+| `response_length` | `None` | Supports `short`, `medium`, `large`, `max`, integer, or numeric string. |
+| `category` | `None` | Provider-specific category. |
+| `start_date` / `end_date` | `None` | Optional `YYYY-MM-DD` bounds. |
+| `max_price` | `None` | Provider cost limit if used. |
+| `fast_mode` | `False` | Sent explicitly. |
+| `url_only` | `False` | Sent explicitly. |
+| `source_biases` | `None` | Optional per-source integer biases. |
+| `instructions` | `None` | Optional provider instructions. |
+
+`is_tool_call` is not exposed to the model; the Valyu provider sends it as
+`True`.
+
+Valyu provider metadata that is not part of the normalized result shape, such as
+scores or ranking fields when returned by the SDK, is preserved in each result's
+`metadata` object.
+
+Custom providers implement `SearchProvider`:
+
+```python
+from modis_tools.providers.web import Page, SearchResult
+
+class MyProvider:
+    def search(self, query: str, *, num_results: int = 10, **kwargs: object) -> list[SearchResult]:
+        return [SearchResult(title="Example", url="https://example.test", content="Page text")]
+
+    def fetch(self, url: str) -> Page:
+        return Page(title="Fetched", url=url, markdown="Fetched markdown")
+```
+
+`web.open` can open prior search results by index/result id/URL, or an arbitrary
+direct URL. Browser-backed fetching is still kept behind the optional `browser`
+extra for future extension.
+
+## Live Web Tests
+
+Live web tests are skipped unless `VALYU_API_KEY` is set and the `web` extra is
+installed:
+
+```bash
+python -m pip install -e ".[dev,web]"
+VALYU_API_KEY=... pytest -m live
+```
+
+The broader web quality eval is opt-in because it consumes live provider quota:
+
+```bash
+VALYU_API_KEY=... MODIS_TOOLS_RUN_WEB_EVAL=1 pytest -m "live and eval" -vv
+```
+
+See `docs/WEB_EVAL.md` for the current query set and comparison checklist.

modis_py_tools-0.1.0/pyproject.toml

@@ -0,0 +1,57 @@
+[build-system]
+requires = ["setuptools>=67", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "modis-py-tools"
+version = "0.1.0"
+description = "Common MoDIS-compatible Python tools and tool-call execution helpers"
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = [
+  "pydantic>=2.7.0"
+]
+
+[project.optional-dependencies]
+web = [
+  "valyu>=2.0.0"
+]
+browser = [
+  "markdownify>=0.12.0",
+  "playwright>=1.45.0",
+  "readability-lxml>=0.8.1"
+]
+dev = [
+  "build>=1.2.0",
+  "mypy>=1.10.0",
+  "pytest>=8.0.0",
+  "ruff>=0.8.0"
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["modis_tools*"]
+
+[tool.setuptools.package-data]
+modis_tools = ["py.typed"]
+
+[tool.ruff]
+line-length = 120
+target-version = "py311"
+
+[tool.ruff.lint]
+select = ["E", "F", "I"]
+
+[tool.mypy]
+python_version = "3.11"
+check_untyped_defs = true
+ignore_missing_imports = true
+no_implicit_optional = true
+warn_unused_configs = true
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+markers = [
+  "live: tests that call live external providers",
+  "eval: opt-in quality evaluation tests that may be slower or provider-costly"
+]

modis_py_tools-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+