@yahyahammad/venastine 0.3.9 → 0.3.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONFIG_ARCHITECTURE.md +109 -0
- package/README.md +13 -4
- package/bin/venastine.mjs +62 -6
- package/bin/venastine.py +3 -1
- package/cli_commands.py +10 -0
- package/config.yaml +51 -0
- package/config_schema.py +16 -0
- package/core/client.py +45 -4
- package/core/commands.py +4 -0
- package/core/provider_errors.py +40 -0
- package/core/reasoning/scholar.py +14 -5
- package/docs/SECURITY.md +6 -0
- package/mcp_client/client.py +49 -2
- package/package.json +1 -1
- package/pyproject.toml +1 -1
- package/requirements.txt +19 -0
- package/tools/builtin/_net_common.py +878 -0
- package/tools/builtin/_page_text.py +490 -0
- package/tools/builtin/_pdf_text.py +113 -0
- package/tools/builtin/arxiv.py +5 -1
- package/tools/builtin/fetch_url.py +388 -120
- package/tools/builtin/web_search.py +81 -7
- package/tools/registry.py +10 -0
- package/tui/app.py +54 -0
- package/tui/app.tcss +7 -0
- package/tui/widgets.py +48 -0
package/docs/SECURITY.md
CHANGED
|
@@ -92,6 +92,12 @@ Note that all of these settings live in `config.yaml` (`shell_approval_mode`, `t
|
|
|
92
92
|
|
|
93
93
|
**What the hosted demo runs is a commit of `main` that passed every CI check, and the deploy holds one token (ROADMAP_v3 §79, HD9, amended in batch 169).** `.github/workflows/deploy-demo.yml` deploys the Hugging Face Space only after `scripts/deploy_gate.py` has read GitHub's record of the commit and found every check run on it success or skipped -- including a job its workflow lets fail and a scheduled run on the same commit -- every workflow a push to `main` runs passed, and the commit still `main`'s tip. The gate's job holds no secret; the deploy job is the only one in the `demo` environment, which holds `HF_TOKEN` (fine-grained, write access to the one Space) and releases it to `main` only. A `workflow_run` job's `github.ref` is always the default branch, so the gate itself refuses anything but a push to `main` in this repository -- a fork's pull request from a branch named `main` included -- and no event value is pasted into a shell line. Before anything is sent, the bundle is scanned by gitleaks from its own root and its locks audited by pip-audit; a finding is accepted only by ID, with its reason, while its version is pinned. The ModelScope Studio is deployed by the owner with the same script and gate, because ModelScope's tokens are account-wide and it offers no trusted CI. **Whoever can merge to `main` decides the code that runs beside every live visitor's sign-in.** And no code scanning alert of security severity high or critical may be open on `main` (batch 170): CodeQL on a push uploads its alerts and fails nothing -- the `CodeQL` check is posted on pull requests only -- so the gate reads the alerts itself, and an alert list it cannot read stops the deploy.
|
|
94
94
|
|
|
95
|
+
**A network limit bounds an allowance, not an attacker (ROADMAP_v3 §53).** With a ceiling set in `config.yaml`, the harness counts what its own network tools, its model-provider calls and its HTTP MCP servers send and receive, and refuses a call that would pass the ceiling. What it CANNOT count is written here rather than left to be discovered (IO5): shell commands -- containers, WSL and SSH open their own sockets, and `--network none` is the control on that path -- `stdio` MCP servers, sign-in and token-refresh traffic, TLS and TCP overhead, and bytes already in a socket buffer when a read stopped early. `web_search`'s figure is a measured estimate, because its client has no hook. A ceiling is not an authority key and not a security boundary. In scope: a route by which model output or repository content raises or bypasses a ceiling the user set, for traffic the meter does count. Out of scope: anything on the list above.
|
|
96
|
+
|
|
97
|
+
**A fetched PDF is parsed in a separate, killable process (ROADMAP_v3 §92, PT5).** PDF parsers have a long history of hostile files that make them loop or exhaust memory, so `fetch_url` never parses one in the agent's process: pypdf runs in the child `tools/isolation.py` starts for the maths tools, under `tool_compute_timeout_s` and, on POSIX, CPU and address-space limits. A PDF over `fetch_max_pdf_mb` is not downloaded. In scope: a PDF or page that makes `fetch_url` hang the agent, read outside its bounds, or hand the model text the redactor never saw.
|
|
98
|
+
|
|
99
|
+
**An installed runtime's files are copies, never links into a package cache (ROADMAP_v3 §93, UV4).** The GitHub installers and the npm launcher build their Python environment under `~/.config/venastine/runtime`, inside the user tier the harness protects from tool writes, and since batch 241 they build it with uv when uv is on PATH. uv's default on Linux and Windows hardlinks every file out of its cache, which would make the harness's imported code the same inode as a file under `~/.cache/uv` -- a directory nothing protects, and one a sandboxed command can write when the workspace contains the home directory. So uv is always told `--link-mode copy` (measured: `st_nlink` 2 under the default, 1 copied; an integration test holds it on both platforms), and a uv that fails hands over to pip, whose installs were always copies.
|
|
100
|
+
|
|
95
101
|
## No bounty
|
|
96
102
|
|
|
97
103
|
This is an independent research project with no funding — **we cannot offer monetary rewards, bounties, or swag.** We will credit you publicly if you want, and we will prioritize a fix. If you need a bounty, please use a third-party program that covers this repo, or treat the report as a voluntary disclosure.
|
package/mcp_client/client.py
CHANGED
|
@@ -57,6 +57,7 @@ from mcp.shared.exceptions import MCPError
|
|
|
57
57
|
|
|
58
58
|
import config as _harness_config
|
|
59
59
|
from safety.policy_enforcement import TOKEN_CHARS, fragment_end
|
|
60
|
+
from tools.builtin import _net_common
|
|
60
61
|
|
|
61
62
|
logger = logging.getLogger(__name__)
|
|
62
63
|
|
|
@@ -143,6 +144,20 @@ _thread: Optional[threading.Thread] = None
|
|
|
143
144
|
_loop_lock = threading.Lock()
|
|
144
145
|
|
|
145
146
|
|
|
147
|
+
def _metered_mcp_client(headers=None, timeout=None, auth=None):
|
|
148
|
+
"""The SDK's own MCP client, with the MCP meter's hooks on it (§53).
|
|
149
|
+
|
|
150
|
+
`sse_client`'s `httpx_client_factory` signature, so the one function
|
|
151
|
+
serves both HTTP transports. The SDK builds the client -- its timeouts,
|
|
152
|
+
its redirect rule -- and `event_hooks` is public on an httpx2 client,
|
|
153
|
+
so nothing here re-states what the SDK already decides.
|
|
154
|
+
"""
|
|
155
|
+
client = create_mcp_http_client(headers=headers, timeout=timeout,
|
|
156
|
+
auth=auth)
|
|
157
|
+
client.event_hooks = _net_common.mcp_event_hooks()
|
|
158
|
+
return client
|
|
159
|
+
|
|
160
|
+
|
|
146
161
|
def _ensure_loop() -> asyncio.AbstractEventLoop:
|
|
147
162
|
"""The one MCP event loop, created on first use."""
|
|
148
163
|
global _loop, _thread
|
|
@@ -425,6 +440,16 @@ class MCPClient:
|
|
|
425
440
|
# create_mcp_http_client defaults -- so an Authorization
|
|
426
441
|
# header authenticates here too, never discarded (M19's
|
|
427
442
|
# property holds on both HTTP transports).
|
|
443
|
+
#
|
|
444
|
+
# ROADMAP_v3 §53: with the MCP meter on, the SDK's own factory
|
|
445
|
+
# builds the client and the meter's hooks go on it, so its
|
|
446
|
+
# timeouts are the SDK's and only the counting is ours.
|
|
447
|
+
if _net_common.MCP.enabled():
|
|
448
|
+
return sse_client(
|
|
449
|
+
cfg.url,
|
|
450
|
+
headers=dict(cfg.headers) or None,
|
|
451
|
+
httpx_client_factory=_metered_mcp_client,
|
|
452
|
+
)
|
|
428
453
|
return sse_client(
|
|
429
454
|
cfg.url,
|
|
430
455
|
headers=dict(cfg.headers) or None,
|
|
@@ -449,10 +474,20 @@ class MCPClient:
|
|
|
449
474
|
# whole (CONFIG_ARCHITECTURE.md has the measurement). The legacy
|
|
450
475
|
# "sse" branch above keeps the SDK's fixed 1 MiB: `sse_client`
|
|
451
476
|
# takes no size.
|
|
477
|
+
#
|
|
478
|
+
# ROADMAP_v3 §53: with the MCP meter on there is ALWAYS a client
|
|
479
|
+
# of ours, headers or not, because the hooks have to be on it.
|
|
480
|
+
# Built by the SDK's helper either way, so its timeouts hold.
|
|
481
|
+
if _net_common.MCP.enabled():
|
|
482
|
+
http_client = _metered_mcp_client(
|
|
483
|
+
headers=dict(cfg.headers) or None)
|
|
484
|
+
elif cfg.headers:
|
|
485
|
+
http_client = create_mcp_http_client(headers=dict(cfg.headers))
|
|
486
|
+
else:
|
|
487
|
+
http_client = None
|
|
452
488
|
return streamable_http_client(
|
|
453
489
|
cfg.url,
|
|
454
|
-
http_client=
|
|
455
|
-
if cfg.headers else None),
|
|
490
|
+
http_client=http_client,
|
|
456
491
|
max_sse_event_size=_harness_config.MCP_MAX_EVENT_BYTES,
|
|
457
492
|
)
|
|
458
493
|
raise ValueError(cfg.error or "unusable server configuration")
|
|
@@ -617,6 +652,18 @@ class MCPClient:
|
|
|
617
652
|
reason = self.failures.get(server_name, "not connected")
|
|
618
653
|
return {"error": f"MCP server {server_name!r} is unavailable: {reason}"}
|
|
619
654
|
|
|
655
|
+
# ROADMAP_v3 §53 (IO11). Admitted HERE, before anything is sent, and
|
|
656
|
+
# never in the HTTP hooks: a refusal raised inside the SDK would
|
|
657
|
+
# surface in its background writer and read as a timeout. Only the
|
|
658
|
+
# servers the meter can see are asked -- a "stdio" server's traffic
|
|
659
|
+
# is its own process's, and IO5 says so rather than pretending.
|
|
660
|
+
cfg = self.server_configs.get(server_name)
|
|
661
|
+
if getattr(cfg, "transport", None) in ("http", "sse"):
|
|
662
|
+
try:
|
|
663
|
+
_net_common.MCP.admit(_net_common.mcp_call_bytes(params))
|
|
664
|
+
except _net_common.NetworkLimitReached as e:
|
|
665
|
+
return {"error": e.reason}
|
|
666
|
+
|
|
620
667
|
loop = _ensure_loop()
|
|
621
668
|
protocol_timeout = max(1.0, timeout - _TIMEOUT_MARGIN_S)
|
|
622
669
|
coro = client.call_tool(tool_name, params,
|
package/package.json
CHANGED
package/pyproject.toml
CHANGED
|
@@ -18,7 +18,7 @@ build-backend = "setuptools.build_meta"
|
|
|
18
18
|
|
|
19
19
|
[project]
|
|
20
20
|
name = "venastine-research-harness"
|
|
21
|
-
version = "0.3.
|
|
21
|
+
version = "0.3.10"
|
|
22
22
|
description = "A from-scratch agentic harness: tool-using chat plus a ten-pass self-critiquing research pipeline."
|
|
23
23
|
readme = "README.md"
|
|
24
24
|
requires-python = ">=3.11"
|
package/requirements.txt
CHANGED
|
@@ -288,6 +288,25 @@ PyJWT>=2.15.0,<3
|
|
|
288
288
|
# parser's refusals are exactly what a silent minor bump could change.
|
|
289
289
|
defusedxml==0.7.1
|
|
290
290
|
|
|
291
|
+
# Batch 240, the owner's choice: fetch_url reads PDFs, and pypdf is what it
|
|
292
|
+
# reads them with (tools/builtin/_pdf_text.py, in tools/isolation's killable
|
|
293
|
+
# child). MEASURED against markitdown[pdf] on three arXiv papers: pypdf kept
|
|
294
|
+
# every probe sentence whole on a one-column and a two-column paper and ran
|
|
295
|
+
# 0.8% of words together on the 77-page Llama 2 report; markitdown ran words
|
|
296
|
+
# together on all three (22% on Llama 2) and turned BERT's two columns into a
|
|
297
|
+
# table of fragments. pypdf is pure Python with no dependencies on 3.11+
|
|
298
|
+
# (4.1 MB); markitdown[pdf] is 188 MB.
|
|
299
|
+
#
|
|
300
|
+
# 6.21.4, NOT the two-weeks-old 6.21.2, the owner's choice: pip-audit found
|
|
301
|
+
# seven advisories in 6.21.2 -- crafted PDFs that loop forever or exhaust
|
|
302
|
+
# memory (CVE-2026-119203 to -119209), which is precisely what a tool reading
|
|
303
|
+
# strangers' PDFs parses -- all fixed by 6.21.4 (2026-10-28), which audits
|
|
304
|
+
# clean and extracted the three papers identically. The rule gives way to a
|
|
305
|
+
# security fix, as it did for mcp 2.2.0 in batch 165. Exact, for defusedxml's
|
|
306
|
+
# reason: a parser of hostile input is where a silent bump changes what is
|
|
307
|
+
# refused.
|
|
308
|
+
pypdf==6.21.4
|
|
309
|
+
|
|
291
310
|
# Test dependencies live in requirements-test.txt, NOT here (S18-07,
|
|
292
311
|
# batch 136). pyproject.toml reads THIS file for `[project].dependencies`
|
|
293
312
|
# via `dynamic`, so anything below this line is a hard requirement of the
|