@allansantos-dev/smart-tool 0.9.8 → 0.9.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -0
- package/NOTICE +3 -0
- package/README.md +20 -2
- package/block_patterns.py +19 -12
- package/browser_control.py +202 -0
- package/endpoint_sync.py +9 -3
- package/install.py +21 -0
- package/install_runtime.py +86 -33
- package/package.json +1 -1
- package/setup_ui.py +219 -46
- package/smart_tool_daemon.py +39 -1
- package/version.py +1 -1
- package/web_adapters/node/package-lock.json +46 -2
- package/web_adapters/node/package.json +1 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,31 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.9.10 - beta
|
|
4
|
+
|
|
5
|
+
- Browser control for agents, on by default: registering smart-tool in Claude Code or Codex (and every update, for
|
|
6
|
+
clients where it is registered) also registers Microsoft's Playwright MCP (`@playwright/mcp` 0.0.83, pinned in
|
|
7
|
+
`web_adapters/node`) as the `playwright` server, driving the Chromium Smart Tool already installs, headless, with an
|
|
8
|
+
isolated in-memory profile. A probe first starts the server and opens a blank page the way an agent would; where
|
|
9
|
+
that fails (a corporate policy against remote debugging, a missing browser, the client CLI refusing) nothing is
|
|
10
|
+
registered, agents keep the read-only browsers of `web_fetch` and `web_search`, and the setup screen shows the
|
|
11
|
+
reason and a retry button instead of an error. A registration Smart Tool made is refreshed on update (Node and
|
|
12
|
+
Chromium paths change) and removed when the probe stops passing; a `playwright` server you registered is kept.
|
|
13
|
+
- Block pattern proposals no longer stack one block per proposal (21 pending ones made the setup screen scroll
|
|
14
|
+
forever): one approval panel shows 8 per page, ordered by how many logged calls each would have blocked, with a
|
|
15
|
+
filter, a select-page toggle and bulk accept/reject in a single write (all ids or none); reason and example open on
|
|
16
|
+
demand, and blocking patterns sit in a collapsed list.
|
|
17
|
+
|
|
18
|
+
## 0.9.9 - beta
|
|
19
|
+
|
|
20
|
+
- `web_fetch` renders JavaScript pages with [Moli](https://github.com/lexmount/moli) first (a headless browser built
|
|
21
|
+
for agents: layout and paint only when a page needs them; 43 MB download, SHA-256 checked) and starts Chromium
|
|
22
|
+
through Crawl4AI only when Moli fails or comes back thin. On 21 pages HTTP could not read: Moli 1.6 s and 90 MB
|
|
23
|
+
peak (median) against 4.2 s and 581 MB; Moli read 14, Chromium 16, both together 17 (only Chromium read YouTube,
|
|
24
|
+
Google Maps and Airbnb). With several agent sessions reading at once, measured on N simultaneous rendered reads:
|
|
25
|
+
aggregate peak memory 448 → 35 MB (1), 1,788 → 194 MB (3), 2,901 → 634 MB (5), 4.4-4.7 GB → 1.0-1.4 GB (10), and
|
|
26
|
+
the median read time halved (10 at once: 5.0 s → 2.5 s). Pages only Chromium reads now take Moli's 1-4 s more.
|
|
27
|
+
The `web_fetch` metric records which engine read the page and Moli's error when it fell back.
|
|
28
|
+
|
|
3
29
|
## 0.9.8 - beta
|
|
4
30
|
|
|
5
31
|
- An update could keep running the previous code: npm dates every file in the package 1985-10-26, the installer kept
|
package/NOTICE
CHANGED
|
@@ -15,6 +15,7 @@ Python packages (installation venv, from PyPI)
|
|
|
15
15
|
|
|
16
16
|
Browser runtime (web_adapters/browser/.venv, from PyPI and the projects' download servers)
|
|
17
17
|
- camoufox Python package (MIT) and the Camoufox browser, a Firefox fork (MPL-2.0), fetched by `camoufox fetch`
|
|
18
|
+
- Moli (Apache-2.0 OR MIT), https://github.com/lexmount/moli: renders JavaScript pages for web_fetch; downloaded at install time from its GitHub release (SHA-256 checked), with its own license files
|
|
18
19
|
- crawl4ai (Apache-2.0), https://github.com/unclecode/crawl4ai: page rendering is powered by Crawl4AI
|
|
19
20
|
- Playwright (Apache-2.0) and the Chromium build it installs (BSD-style, see https://www.chromium.org)
|
|
20
21
|
|
|
@@ -24,6 +25,8 @@ Node runtime and packages (web_adapters/node, from nodejs.org and npm)
|
|
|
24
25
|
- typescript (Apache-2.0)
|
|
25
26
|
- java-parser (Apache-2.0)
|
|
26
27
|
- @angular/compiler (MIT)
|
|
28
|
+
- @playwright/mcp (Apache-2.0), https://github.com/microsoft/playwright-mcp: browser control for agents, registered as
|
|
29
|
+
the `playwright` MCP server over the Chromium above
|
|
27
30
|
- their dependencies as pinned in web_adapters/node/package-lock.json (MIT, ISC, BSD and Apache-2.0)
|
|
28
31
|
|
|
29
32
|
Web search sources
|
package/README.md
CHANGED
|
@@ -9,6 +9,9 @@ A local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharpe
|
|
|
9
9
|
| `web_fetch` | Reads a known URL and returns only the answer to your prompt, written by a small model; pages are cached for 24 h. Pages over 60,000 characters are not cut: the model reads the parts closest to the prompt (needs an `embedding_model`) |
|
|
10
10
|
| `project_manage` | Registers and indexes projects, shows coverage, code maps, duplicated functions and index storage; `graph` with `symbol` tells what changing a function touches: callers, calls, the tests that reach it and the files that import it |
|
|
11
11
|
|
|
12
|
+
Agents that must click, type or log in also get Microsoft's [Playwright MCP](https://github.com/microsoft/playwright-mcp),
|
|
13
|
+
registered as the `playwright` server where this machine lets a browser be driven (see Set up, step 3).
|
|
14
|
+
|
|
12
15
|
An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
|
|
13
16
|
are cheaper, or just tells the agent what Smart Tool would do.
|
|
14
17
|
|
|
@@ -51,7 +54,18 @@ Open the setup screen from the tray icon ("Open settings") or at `http://127.0.0
|
|
|
51
54
|
reasoning: web search tier order and result checks), the index classification model (also reviews block patterns) and the research model. Any id the gateway
|
|
52
55
|
accepts works; the catalog is shown as suggestions.
|
|
53
56
|
3. **Agents**: register the MCP server in Claude Code or Codex (runs their official `mcp add` command), install the
|
|
54
|
-
hook and choose its behavior
|
|
57
|
+
hook and choose its behavior.
|
|
58
|
+
|
|
59
|
+
**Browser control** comes with the registration, and with every update for clients where smart-tool is registered:
|
|
60
|
+
the official Playwright MCP (pinned in `web_adapters/node`) is registered as the `playwright` server, driving the
|
|
61
|
+
Chromium Smart Tool installs, headless, with an isolated in-memory profile (no logins of yours) and its files in
|
|
62
|
+
`~/.smart-tool/data/playwright-mcp`. Before registering, Smart Tool starts that server and opens a blank page as an
|
|
63
|
+
agent would; where that fails (a corporate policy that forbids remote debugging, a missing browser) nothing is
|
|
64
|
+
registered, agents keep reading pages through `web_fetch`, and the setup screen shows the reason with a retry
|
|
65
|
+
button. A `playwright` server you registered yourself is never touched. Remove it with
|
|
66
|
+
`claude mcp remove playwright -s user` / `codex mcp remove playwright`.
|
|
67
|
+
|
|
68
|
+
The hook behaviors:
|
|
55
69
|
- **Redirect** (default): the native tool is denied with the reason and the Smart Tool tool to use. The decision
|
|
56
70
|
is a fixed rule, under a millisecond and the same every time: a content search over a project folder (`grep -r`,
|
|
57
71
|
`rg`, `git grep`, the Grep tool on a folder) or over more than 20 listed files goes to `smart_search`; reading or
|
|
@@ -73,6 +87,10 @@ Open the setup screen from the tray icon ("Open settings") or at `http://127.0.0
|
|
|
73
87
|
A documented function edited only in its body is not mentioned. `project_manage` with `action=docs` lists the
|
|
74
88
|
public functions still without a docstring, for documenting a project that started without them.
|
|
75
89
|
|
|
90
|
+
**Block patterns waiting for you** are shown in one approval panel: 8 per page, the ones that would have blocked
|
|
91
|
+
the most logged calls first, a filter over pattern, reason and example, and bulk accept or reject of the selected
|
|
92
|
+
rows; each row opens to its reason and example.
|
|
93
|
+
|
|
76
94
|
**Duplicate functions on edit** compares the functions an edit writes with the indexed code. **Warn** (default)
|
|
77
95
|
tells the agent when a written function has the same body as an existing one (comments, spacing and docstrings
|
|
78
96
|
ignored) or a near-identical one (only local names, strings and numbers changed), with the path and lines of the
|
|
@@ -146,4 +164,4 @@ user` / `codex mcp remove smart-tool`, then delete `%LOCALAPPDATA%\Programs\Smar
|
|
|
146
164
|
|
|
147
165
|
Apache License 2.0. See `LICENSE` and `NOTICE` for third-party components.
|
|
148
166
|
|
|
149
|
-
Page rendering is powered by [Crawl4AI](https://github.com/unclecode/crawl4ai) (Apache-2.0).
|
|
167
|
+
Page rendering is powered by [Moli](https://github.com/lexmount/moli) (Apache-2.0 / MIT), with [Crawl4AI](https://github.com/unclecode/crawl4ai) (Apache-2.0) as its fallback.
|
package/block_patterns.py
CHANGED
|
@@ -247,20 +247,27 @@ def review_later(tool_name, tool_input, rule_reason, metrics_path, redact, log):
|
|
|
247
247
|
return True
|
|
248
248
|
|
|
249
249
|
|
|
250
|
-
def
|
|
251
|
-
"""accept moves
|
|
250
|
+
def decide_proposals(entry_ids, action):
|
|
251
|
+
"""accept moves proposals to accepted; reject moves them to rejected; remove deletes accepted patterns. All ids are
|
|
252
|
+
decided in one write, or none when any id is unknown (ValueError naming it)."""
|
|
253
|
+
if action not in ("accept", "reject", "remove"):
|
|
254
|
+
raise ValueError("Invalid action: use accept, reject or remove.")
|
|
255
|
+
wanted = list(dict.fromkeys(entry_ids))
|
|
256
|
+
if not wanted:
|
|
257
|
+
raise ValueError("No pattern selected.")
|
|
252
258
|
with _LOCK:
|
|
253
259
|
data = load()
|
|
254
260
|
source = "accepted" if action == "remove" else "proposals"
|
|
255
|
-
|
|
256
|
-
if
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
261
|
+
found = {e["id"]: e for e in data[source] if e["id"] in wanted}
|
|
262
|
+
missing = [entry_id for entry_id in wanted if entry_id not in found]
|
|
263
|
+
if missing:
|
|
264
|
+
raise ValueError(f"No {source[:-1]} with id {', '.join(missing)}.")
|
|
265
|
+
data[source] = [e for e in data[source] if e["id"] not in found]
|
|
266
|
+
now = time.time()
|
|
267
|
+
for entry_id in wanted:
|
|
268
|
+
if action == "accept":
|
|
269
|
+
data["accepted"].append({**found[entry_id], "accepted_at": now})
|
|
270
|
+
elif action == "reject":
|
|
271
|
+
data["rejected"].append({**found[entry_id], "rejected_by": "user"})
|
|
265
272
|
_save(data)
|
|
266
273
|
return data
|
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
"""Browser control for agents: the official Playwright MCP server, registered by default next to smart-tool.
|
|
2
|
+
|
|
3
|
+
Smart Tool's own browsers only read pages (web_fetch, web_search). Agents that must click, type or log in get
|
|
4
|
+
Microsoft's Playwright MCP (pinned in web_adapters/node), driving the Chromium Smart Tool already installs, in an
|
|
5
|
+
isolated in-memory profile, headless. Before registering, a probe launches that browser through the server exactly
|
|
6
|
+
as the agent would: machines whose policy forbids it (corporate IT blocking remote debugging, a missing browser)
|
|
7
|
+
keep today's read-only behavior, and the reason is shown in the setup screen instead of an error. A server named
|
|
8
|
+
`playwright` the user already registered is left untouched.
|
|
9
|
+
"""
|
|
10
|
+
import json
|
|
11
|
+
import os
|
|
12
|
+
import queue
|
|
13
|
+
import subprocess
|
|
14
|
+
import tempfile
|
|
15
|
+
import threading
|
|
16
|
+
import time
|
|
17
|
+
|
|
18
|
+
import atomic_io
|
|
19
|
+
import endpoint_sync
|
|
20
|
+
import install_runtime
|
|
21
|
+
import paths
|
|
22
|
+
|
|
23
|
+
SERVER = "playwright"
|
|
24
|
+
CLI = install_runtime.PLAYWRIGHT_MCP_CLI
|
|
25
|
+
STATUS_PATH = os.path.join(paths.DATA_DIR, "browser-control.json")
|
|
26
|
+
OUTPUT_DIR = os.path.join(paths.DATA_DIR, "playwright-mcp")
|
|
27
|
+
OUTPUT_MAX_BYTES = 100 * 1024 * 1024
|
|
28
|
+
PROBE_TIMEOUT_S = 60
|
|
29
|
+
_CHROMIUM_PROBE = ("from playwright.sync_api import sync_playwright\n"
|
|
30
|
+
"with sync_playwright() as p: print(p.chromium.executable_path)")
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
def chromium_executable():
|
|
34
|
+
"""The Chromium that `playwright install chromium` put in place for the browser venv; RuntimeError when absent."""
|
|
35
|
+
if not install_runtime.BROWSER_PYTHON.is_file():
|
|
36
|
+
raise RuntimeError(f"Browser runtime missing ({install_runtime.BROWSER_PYTHON}).")
|
|
37
|
+
done = subprocess.run([str(install_runtime.BROWSER_PYTHON), "-c", _CHROMIUM_PROBE], capture_output=True,
|
|
38
|
+
text=True, encoding="utf-8", errors="replace", timeout=60,
|
|
39
|
+
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0))
|
|
40
|
+
path = (done.stdout.strip().splitlines() or [""])[-1]
|
|
41
|
+
if done.returncode or not os.path.isfile(path):
|
|
42
|
+
raise RuntimeError(f"Chromium not found: {(done.stderr or done.stdout).strip()[-300:] or path}")
|
|
43
|
+
return path
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
def server_command():
|
|
47
|
+
"""Absolute command line of the Playwright MCP server, as registered in the clients."""
|
|
48
|
+
node, _npm = install_runtime._node_commands()
|
|
49
|
+
if not node:
|
|
50
|
+
raise RuntimeError("Supported Node.js not found for Playwright MCP.")
|
|
51
|
+
if not CLI.is_file():
|
|
52
|
+
raise RuntimeError(f"Playwright MCP is not installed ({CLI}); reinstall Smart Tool.")
|
|
53
|
+
return [node, str(CLI), "--isolated", "--headless", "--executable-path", chromium_executable(),
|
|
54
|
+
"--output-dir", OUTPUT_DIR, "--output-max-size", str(OUTPUT_MAX_BYTES)]
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def probe(command, timeout=PROBE_TIMEOUT_S):
|
|
58
|
+
"""Starts the server over stdio, opens about:blank and closes it, as an agent's first call would.
|
|
59
|
+
Returns None when the browser opened, else the reason it did not (a policy block, a crash, a timeout)."""
|
|
60
|
+
with tempfile.TemporaryDirectory(prefix="st-browser-probe-", ignore_cleanup_errors=True) as workdir:
|
|
61
|
+
return _probe(command, workdir, timeout)
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def _probe(command, workdir, timeout):
|
|
65
|
+
"""probe() inside workdir, the server's working folder."""
|
|
66
|
+
flags = getattr(subprocess, "CREATE_NO_WINDOW", 0)
|
|
67
|
+
try:
|
|
68
|
+
proc = subprocess.Popen(command, cwd=workdir, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
|
69
|
+
stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace",
|
|
70
|
+
creationflags=flags)
|
|
71
|
+
except OSError as exc:
|
|
72
|
+
return f"server did not start: {exc}"
|
|
73
|
+
lines, errors = queue.Queue(), []
|
|
74
|
+
threading.Thread(target=lambda: [lines.put(line) for line in proc.stdout] + [lines.put(None)], daemon=True).start()
|
|
75
|
+
threading.Thread(target=lambda: errors.extend(proc.stderr), daemon=True).start()
|
|
76
|
+
deadline = time.monotonic() + timeout
|
|
77
|
+
|
|
78
|
+
def call(request_id, method, params):
|
|
79
|
+
proc.stdin.write(json.dumps({"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}) + "\n")
|
|
80
|
+
proc.stdin.flush()
|
|
81
|
+
while True:
|
|
82
|
+
try:
|
|
83
|
+
line = lines.get(timeout=max(0.1, deadline - time.monotonic()))
|
|
84
|
+
except queue.Empty:
|
|
85
|
+
raise TimeoutError(f"no answer to {method} in {timeout}s") from None
|
|
86
|
+
if line is None:
|
|
87
|
+
raise RuntimeError(f"server exited: {''.join(errors).strip()[-300:] or 'no output'}")
|
|
88
|
+
try:
|
|
89
|
+
message = json.loads(line)
|
|
90
|
+
except ValueError:
|
|
91
|
+
continue
|
|
92
|
+
if message.get("id") == request_id:
|
|
93
|
+
if "error" in message:
|
|
94
|
+
raise RuntimeError(str(message["error"].get("message") or message["error"])[:300])
|
|
95
|
+
return message["result"]
|
|
96
|
+
|
|
97
|
+
try:
|
|
98
|
+
call(1, "initialize", {"protocolVersion": "2025-06-18", "capabilities": {},
|
|
99
|
+
"clientInfo": {"name": "smart-tool-probe", "version": "1"}})
|
|
100
|
+
proc.stdin.write(json.dumps({"jsonrpc": "2.0", "method": "notifications/initialized"}) + "\n")
|
|
101
|
+
result = call(2, "tools/call", {"name": "browser_navigate", "arguments": {"url": "about:blank"}})
|
|
102
|
+
if result.get("isError"):
|
|
103
|
+
text = " ".join(part.get("text", "") for part in result.get("content") or [])
|
|
104
|
+
return f"browser did not open: {' '.join(text.split())[:300]}"
|
|
105
|
+
call(3, "tools/call", {"name": "browser_close", "arguments": {}})
|
|
106
|
+
return None
|
|
107
|
+
except (OSError, RuntimeError, TimeoutError) as exc:
|
|
108
|
+
return str(exc)
|
|
109
|
+
finally:
|
|
110
|
+
if os.name == "nt":
|
|
111
|
+
subprocess.run(["taskkill", "/PID", str(proc.pid), "/T", "/F"], capture_output=True, creationflags=flags)
|
|
112
|
+
else:
|
|
113
|
+
proc.kill()
|
|
114
|
+
proc.wait(timeout=10)
|
|
115
|
+
for stream in (proc.stdin, proc.stdout, proc.stderr):
|
|
116
|
+
try:
|
|
117
|
+
stream.close()
|
|
118
|
+
except OSError:
|
|
119
|
+
pass
|
|
120
|
+
|
|
121
|
+
|
|
122
|
+
def registered():
|
|
123
|
+
"""Whether each client already has a `playwright` MCP server (registered by Smart Tool or by the user)."""
|
|
124
|
+
found = {"claude": False, "codex": False}
|
|
125
|
+
try:
|
|
126
|
+
with open(endpoint_sync.claude_json(), encoding="utf-8") as stream:
|
|
127
|
+
found["claude"] = SERVER in (json.load(stream).get("mcpServers") or {})
|
|
128
|
+
except (OSError, ValueError, AttributeError):
|
|
129
|
+
pass
|
|
130
|
+
try:
|
|
131
|
+
with open(os.path.join(os.path.expanduser("~"), ".codex", "config.toml"), encoding="utf-8") as stream:
|
|
132
|
+
found["codex"] = f"[mcp_servers.{SERVER}]" in stream.read()
|
|
133
|
+
except OSError:
|
|
134
|
+
pass
|
|
135
|
+
return found
|
|
136
|
+
|
|
137
|
+
|
|
138
|
+
def register_command(client, command):
|
|
139
|
+
"""Official CLI command that registers a stdio server (Claude Code: user scope, every project)."""
|
|
140
|
+
if client == "claude":
|
|
141
|
+
return ["claude", "mcp", "add", "--scope", "user", SERVER, "--", *command]
|
|
142
|
+
if client == "codex":
|
|
143
|
+
return ["codex", "mcp", "add", SERVER, "--", *command]
|
|
144
|
+
raise ValueError(f"Client without MCP registration: {client!r}")
|
|
145
|
+
|
|
146
|
+
|
|
147
|
+
def remove_command(client):
|
|
148
|
+
"""Official CLI command that removes the server registered by register_command."""
|
|
149
|
+
if client == "claude":
|
|
150
|
+
return ["claude", "mcp", "remove", SERVER, "--scope", "user"]
|
|
151
|
+
if client == "codex":
|
|
152
|
+
return ["codex", "mcp", "remove", SERVER]
|
|
153
|
+
raise ValueError(f"Client without MCP registration: {client!r}")
|
|
154
|
+
|
|
155
|
+
|
|
156
|
+
def status():
|
|
157
|
+
"""Last outcome per client: {"claude": {"state": "registered"|"unavailable"|"kept", "reason", "at"}, ...}."""
|
|
158
|
+
try:
|
|
159
|
+
with open(STATUS_PATH, encoding="utf-8") as stream:
|
|
160
|
+
data = json.load(stream)
|
|
161
|
+
except FileNotFoundError:
|
|
162
|
+
return {}
|
|
163
|
+
if not isinstance(data, dict):
|
|
164
|
+
raise ValueError(f"{STATUS_PATH} is not a browser control status file: fix or delete it.")
|
|
165
|
+
return data
|
|
166
|
+
|
|
167
|
+
|
|
168
|
+
def _record(client, state, reason="", log=None):
|
|
169
|
+
if log and reason:
|
|
170
|
+
log(f"browser control {state} for {client}: {reason}")
|
|
171
|
+
data = status()
|
|
172
|
+
data[client] = {"state": state, "reason": reason, "at": time.time()}
|
|
173
|
+
os.makedirs(os.path.dirname(STATUS_PATH), exist_ok=True)
|
|
174
|
+
atomic_io.write_secret_text(STATUS_PATH, json.dumps(data, indent=1, ensure_ascii=False))
|
|
175
|
+
return data[client]
|
|
176
|
+
|
|
177
|
+
|
|
178
|
+
def ensure(client, log=None):
|
|
179
|
+
"""Registers Playwright MCP in the client, or refreshes the registration Smart Tool made before (Node, Chromium
|
|
180
|
+
and install paths change between versions). A `playwright` server the user registered is kept. When the browser
|
|
181
|
+
cannot be driven on this machine (probe failed, CLI refused) records `unavailable` with the reason, removes a
|
|
182
|
+
registration of ours that would no longer work and returns: agents keep the read-only browsers of web_fetch and
|
|
183
|
+
web_search. Only an unreadable status file raises (ValueError). log (optional) also gets the reason."""
|
|
184
|
+
ours = status().get(client, {}).get("state") == "registered"
|
|
185
|
+
if registered().get(client) and not ours:
|
|
186
|
+
return _record(client, "kept", "a playwright MCP server you registered is kept")
|
|
187
|
+
try:
|
|
188
|
+
command = server_command()
|
|
189
|
+
problem = probe(command)
|
|
190
|
+
except (OSError, RuntimeError, subprocess.SubprocessError) as exc:
|
|
191
|
+
problem = str(exc)
|
|
192
|
+
try:
|
|
193
|
+
if ours and registered().get(client):
|
|
194
|
+
endpoint_sync.run_cli(remove_command(client))
|
|
195
|
+
if problem:
|
|
196
|
+
return _record(client, "unavailable", problem[:300], log)
|
|
197
|
+
endpoint_sync.run_cli(register_command(client, command))
|
|
198
|
+
except RuntimeError as exc:
|
|
199
|
+
return _record(client, "unavailable", str(exc)[:300], log)
|
|
200
|
+
if not registered().get(client):
|
|
201
|
+
return _record(client, "unavailable", "the client CLI finished, but the server did not appear in its configuration", log)
|
|
202
|
+
return _record(client, "registered")
|
package/endpoint_sync.py
CHANGED
|
@@ -117,6 +117,15 @@ def register_command(client, base):
|
|
|
117
117
|
def register(client, base):
|
|
118
118
|
"""Runs the client's own CLI (it owns its config file; ~/.claude.json is rewritten by Claude Code all the time)."""
|
|
119
119
|
command = register_command(client, base)
|
|
120
|
+
run_cli(command)
|
|
121
|
+
if registrations().get(client) != f"{base}/mcp":
|
|
122
|
+
raise RuntimeError(f"'{' '.join(command)}' finished, but the registration did not appear in the client's configuration.")
|
|
123
|
+
return " ".join(command)
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
def run_cli(command):
|
|
127
|
+
"""Runs a client CLI command (claude, codex) found on PATH, killing its whole shim tree after REGISTER_TIMEOUT_S.
|
|
128
|
+
Raises RuntimeError with the command and the CLI's own message when it is missing, hangs or exits non-zero."""
|
|
120
129
|
executable = shutil.which(command[0])
|
|
121
130
|
if not executable:
|
|
122
131
|
raise RuntimeError(f"CLI '{command[0]}' not found in Smart Tool's PATH. Register it manually: {' '.join(command)}")
|
|
@@ -140,9 +149,6 @@ def register(client, base):
|
|
|
140
149
|
if proc.returncode != 0:
|
|
141
150
|
detail = (stderr or stdout).strip()[-400:]
|
|
142
151
|
raise RuntimeError(f"'{' '.join(command)}' failed (code {proc.returncode}): {detail}")
|
|
143
|
-
if registrations().get(client) != f"{base}/mcp":
|
|
144
|
-
raise RuntimeError(f"'{' '.join(command)}' finished, but the registration did not appear in the client's configuration.")
|
|
145
|
-
return " ".join(command)
|
|
146
152
|
|
|
147
153
|
|
|
148
154
|
def known_port():
|
package/install.py
CHANGED
|
@@ -291,6 +291,25 @@ def verify(target, python):
|
|
|
291
291
|
raise RuntimeError(f"The active daemon (pid {pid}) does not belong to this installation: {command.strip()[:200]}")
|
|
292
292
|
|
|
293
293
|
|
|
294
|
+
def enable_browser_control(target, python):
|
|
295
|
+
"""Registers Playwright MCP (browser_control.ensure) in each client where smart-tool is registered. Never fails the
|
|
296
|
+
installation: a machine that cannot drive a browser keeps the read-only browsers, and the outcome is printed."""
|
|
297
|
+
script = ("import json, browser_control, endpoint_sync; "
|
|
298
|
+
"print(json.dumps({c: browser_control.ensure(c) for c, url in endpoint_sync.registrations().items() if url}))")
|
|
299
|
+
try:
|
|
300
|
+
result = subprocess.run([str(python), "-c", script], cwd=target, capture_output=True, text=True,
|
|
301
|
+
encoding="utf-8", errors="replace", timeout=300)
|
|
302
|
+
outcome = json.loads(result.stdout.strip().splitlines()[-1]) if not result.returncode else None
|
|
303
|
+
except (OSError, subprocess.TimeoutExpired, ValueError, IndexError) as exc:
|
|
304
|
+
result, outcome = None, f"{type(exc).__name__}: {exc}"
|
|
305
|
+
if not isinstance(outcome, dict):
|
|
306
|
+
detail = outcome or ((result.stderr or result.stdout).strip().splitlines() or ["no output"])[-1]
|
|
307
|
+
print(f"Browser control not set up: {detail}")
|
|
308
|
+
return
|
|
309
|
+
for client, state in outcome.items():
|
|
310
|
+
print(f"Browser control in {client}: {state['state']}" + (f" ({state['reason']})" if state.get("reason") else ""))
|
|
311
|
+
|
|
312
|
+
|
|
294
313
|
MIN_PYTHON = (3, 11)
|
|
295
314
|
|
|
296
315
|
|
|
@@ -363,6 +382,8 @@ def main():
|
|
|
363
382
|
shutil.rmtree(backup, ignore_errors=True)
|
|
364
383
|
raise SystemExit(1)
|
|
365
384
|
shutil.rmtree(backup)
|
|
385
|
+
if not args.no_start:
|
|
386
|
+
enable_browser_control(target, python)
|
|
366
387
|
print("Smart Tool ready." if not args.no_start else "Files installed; daemon not restarted (--no-start).")
|
|
367
388
|
|
|
368
389
|
|
package/install_runtime.py
CHANGED
|
@@ -30,7 +30,17 @@ OPEN_WEBSEARCH_CLI = NODE_DIR / "node_modules" / "open-websearch" / "build" / "i
|
|
|
30
30
|
TYPESCRIPT_PARSER = NODE_DIR / "node_modules" / "typescript" / "lib" / "typescript.js"
|
|
31
31
|
JAVA_PARSER = NODE_DIR / 'node_modules' / 'java-parser' / 'src' / 'index.js'
|
|
32
32
|
ANGULAR_PARSER = NODE_DIR / 'node_modules' / '@angular' / 'compiler' / 'fesm2022' / 'compiler.mjs'
|
|
33
|
+
# Browser control for agents (browser_control.py): Microsoft's Playwright MCP over the Chromium installed below.
|
|
34
|
+
PLAYWRIGHT_MCP_CLI = NODE_DIR / "node_modules" / "@playwright" / "mcp" / "cli.js"
|
|
33
35
|
NODE_RUNTIME = NODE_DIR / ".node-runtime"
|
|
36
|
+
# Moli renders the JavaScript pages web_fetch cannot read over HTTP; Chromium (Crawl4AI) stays as its fallback.
|
|
37
|
+
# Measured on 2026-10-10 over 21 such pages: 1.6 s and 90 MB peak (median) against 4.2 s and 581 MB for Chromium.
|
|
38
|
+
MOLI_VERSION = "1.1.15"
|
|
39
|
+
MOLI_FOLDER = f"moli-v{MOLI_VERSION}-x86_64-pc-windows-msvc"
|
|
40
|
+
MOLI_URL = f"https://github.com/lexmount/moli/releases/download/v{MOLI_VERSION}/moli-x86_64-pc-windows-msvc.zip"
|
|
41
|
+
MOLI_SHA256 = "64c00ce02d6db8e1c22c55e88fbcaaf3e7517e930f24c25ff46a457b421040c8"
|
|
42
|
+
MOLI_DIR = BROWSER_DIR / "moli"
|
|
43
|
+
MOLI_EXE = MOLI_DIR / "moli.exe"
|
|
34
44
|
|
|
35
45
|
|
|
36
46
|
def _node_supported(node):
|
|
@@ -58,6 +68,49 @@ def _node_commands():
|
|
|
58
68
|
return (node, npm) if node and npm and _node_supported(node) else (None, None)
|
|
59
69
|
|
|
60
70
|
|
|
71
|
+
def _download_verified(url, expected_sha256, path):
|
|
72
|
+
"""Downloads url to path and fails when its SHA-256 differs from the published one."""
|
|
73
|
+
digest = hashlib.sha256()
|
|
74
|
+
with urllib.request.urlopen(url, timeout=60) as response, path.open("wb") as out:
|
|
75
|
+
while block := response.read(1024 * 1024):
|
|
76
|
+
digest.update(block)
|
|
77
|
+
out.write(block)
|
|
78
|
+
if digest.hexdigest().lower() != expected_sha256.lower():
|
|
79
|
+
raise RuntimeError(f"SHA-256 of {url} does not match the published checksum.")
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
def _extract_folder(zip_path, top, destination):
|
|
83
|
+
"""Extracts the files under the archive's single top folder into destination, refusing any other path."""
|
|
84
|
+
destination.mkdir()
|
|
85
|
+
with zipfile.ZipFile(zip_path) as zf:
|
|
86
|
+
for member in zf.infolist():
|
|
87
|
+
parts = Path(member.filename.replace("\\", "/")).parts
|
|
88
|
+
if parts == (top,) and member.is_dir():
|
|
89
|
+
continue
|
|
90
|
+
if len(parts) < 2 or parts[0] != top or ".." in parts:
|
|
91
|
+
raise RuntimeError(f"{zip_path.name} contains an unexpected path: {member.filename}")
|
|
92
|
+
target = destination.joinpath(*parts[1:])
|
|
93
|
+
if member.is_dir():
|
|
94
|
+
target.mkdir(parents=True, exist_ok=True)
|
|
95
|
+
else:
|
|
96
|
+
target.parent.mkdir(parents=True, exist_ok=True)
|
|
97
|
+
with zf.open(member) as source, target.open("wb") as out:
|
|
98
|
+
shutil.copyfileobj(source, out)
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
def _replace_folder(unpacked, final, temp_dir):
|
|
102
|
+
"""Swaps final for unpacked, putting the previous folder back if the swap fails."""
|
|
103
|
+
previous = temp_dir / "previous"
|
|
104
|
+
if final.exists():
|
|
105
|
+
os.replace(final, previous)
|
|
106
|
+
try:
|
|
107
|
+
os.replace(unpacked, final)
|
|
108
|
+
except OSError:
|
|
109
|
+
if previous.exists() and not final.exists():
|
|
110
|
+
os.replace(previous, final)
|
|
111
|
+
raise
|
|
112
|
+
|
|
113
|
+
|
|
61
114
|
def _install_node():
|
|
62
115
|
"""Baixa o Node LTS oficial e confere SHA-256 antes de extrair."""
|
|
63
116
|
print("Downloading Node LTS for the Smart Tool web runtime...", flush=True)
|
|
@@ -79,43 +132,38 @@ def _install_node():
|
|
|
79
132
|
with tempfile.TemporaryDirectory(prefix="node-setup-", dir=NODE_DIR) as temp:
|
|
80
133
|
temp_dir = Path(temp)
|
|
81
134
|
zip_path = temp_dir / archive
|
|
82
|
-
|
|
83
|
-
with urllib.request.urlopen(base + archive, timeout=60) as response, zip_path.open("wb") as out:
|
|
84
|
-
while block := response.read(1024 * 1024):
|
|
85
|
-
digest.update(block)
|
|
86
|
-
out.write(block)
|
|
87
|
-
if digest.hexdigest().lower() != expected.lower():
|
|
88
|
-
raise RuntimeError("Node SHA-256 checksum does not match the official index.")
|
|
135
|
+
_download_verified(base + archive, expected, zip_path)
|
|
89
136
|
unpacked = temp_dir / "runtime"
|
|
90
|
-
unpacked
|
|
91
|
-
with zipfile.ZipFile(zip_path) as zf:
|
|
92
|
-
for member in zf.infolist():
|
|
93
|
-
parts = Path(member.filename.replace("\\", "/")).parts
|
|
94
|
-
if parts == (f"node-{version}-win-x64",) and member.is_dir():
|
|
95
|
-
continue
|
|
96
|
-
if len(parts) < 2 or parts[0] != f"node-{version}-win-x64" or ".." in parts:
|
|
97
|
-
raise RuntimeError("Node archive contains an unexpected path.")
|
|
98
|
-
destination = unpacked.joinpath(*parts[1:])
|
|
99
|
-
if member.is_dir():
|
|
100
|
-
destination.mkdir(parents=True, exist_ok=True)
|
|
101
|
-
else:
|
|
102
|
-
destination.parent.mkdir(parents=True, exist_ok=True)
|
|
103
|
-
with zf.open(member) as source, destination.open("wb") as target:
|
|
104
|
-
shutil.copyfileobj(source, target)
|
|
137
|
+
_extract_folder(zip_path, f"node-{version}-win-x64", unpacked)
|
|
105
138
|
if not (unpacked / "node.exe").is_file() or not (unpacked / "npm.cmd").is_file():
|
|
106
139
|
raise RuntimeError("Node package does not contain the expected executables.")
|
|
107
|
-
|
|
108
|
-
if NODE_RUNTIME.exists():
|
|
109
|
-
os.replace(NODE_RUNTIME, previous)
|
|
110
|
-
try:
|
|
111
|
-
os.replace(unpacked, NODE_RUNTIME)
|
|
112
|
-
except OSError:
|
|
113
|
-
if previous.exists() and not NODE_RUNTIME.exists():
|
|
114
|
-
os.replace(previous, NODE_RUNTIME)
|
|
115
|
-
raise
|
|
140
|
+
_replace_folder(unpacked, NODE_RUNTIME, temp_dir)
|
|
116
141
|
print(f"Node {version} installed inside Smart Tool.")
|
|
117
142
|
|
|
118
143
|
|
|
144
|
+
def _moli_ready():
|
|
145
|
+
try:
|
|
146
|
+
return MOLI_EXE.is_file() and (MOLI_DIR / "VERSION").read_text(encoding="utf-8").strip() == MOLI_VERSION
|
|
147
|
+
except OSError:
|
|
148
|
+
return False
|
|
149
|
+
|
|
150
|
+
|
|
151
|
+
def _install_moli():
|
|
152
|
+
"""Downloads the pinned Moli release from GitHub, checks its SHA-256 and unpacks it into web_adapters/browser."""
|
|
153
|
+
print(f"Downloading Moli {MOLI_VERSION} (structured-first headless browser, 43 MB)...", flush=True)
|
|
154
|
+
BROWSER_DIR.mkdir(parents=True, exist_ok=True)
|
|
155
|
+
with tempfile.TemporaryDirectory(prefix="moli-setup-", dir=BROWSER_DIR) as temp:
|
|
156
|
+
temp_dir = Path(temp)
|
|
157
|
+
zip_path = temp_dir / "moli.zip"
|
|
158
|
+
_download_verified(MOLI_URL, MOLI_SHA256, zip_path)
|
|
159
|
+
unpacked = temp_dir / "moli"
|
|
160
|
+
_extract_folder(zip_path, MOLI_FOLDER, unpacked)
|
|
161
|
+
if not (unpacked / "moli.exe").is_file():
|
|
162
|
+
raise RuntimeError("The Moli package does not contain moli.exe.")
|
|
163
|
+
_replace_folder(unpacked, MOLI_DIR, temp_dir)
|
|
164
|
+
print(f"Moli {MOLI_VERSION} installed inside Smart Tool.")
|
|
165
|
+
|
|
166
|
+
|
|
119
167
|
def _run(args, *, cwd=None, label="Web dependency", idle_s=600, stream=False, attempts=1):
|
|
120
168
|
"""Runs a step with UTF-8 output and closed stdin (a prompt fails at once instead of hanging); it is stopped only
|
|
121
169
|
after idle_s without any output, and stream shows the output live."""
|
|
@@ -193,10 +241,12 @@ def _check():
|
|
|
193
241
|
"typescript_parser": bool(node and TYPESCRIPT_PARSER.is_file()),
|
|
194
242
|
"java_parser":bool(node and JAVA_PARSER.is_file()),
|
|
195
243
|
"angular_parser":bool(node and ANGULAR_PARSER.is_file()),
|
|
244
|
+
"playwright_mcp": bool(node and PLAYWRIGHT_MCP_CLI.is_file()),
|
|
196
245
|
"browser_python": BROWSER_PYTHON.is_file(),
|
|
197
246
|
"browser_packages": False,
|
|
198
247
|
"camoufox_browser": False,
|
|
199
248
|
"chromium_browser": False,
|
|
249
|
+
"moli_browser": _moli_ready(),
|
|
200
250
|
}
|
|
201
251
|
if status["browser_python"]:
|
|
202
252
|
try:
|
|
@@ -262,7 +312,8 @@ def install(reuse_browser_runtime=None):
|
|
|
262
312
|
if not node or not npm:
|
|
263
313
|
_install_node()
|
|
264
314
|
node, npm = _node_commands()
|
|
265
|
-
if not all(path.is_file() for path in (OPEN_WEBSEARCH_CLI,TYPESCRIPT_PARSER,JAVA_PARSER,ANGULAR_PARSER
|
|
315
|
+
if not all(path.is_file() for path in (OPEN_WEBSEARCH_CLI, TYPESCRIPT_PARSER, JAVA_PARSER, ANGULAR_PARSER,
|
|
316
|
+
PLAYWRIGHT_MCP_CLI)):
|
|
266
317
|
_run([npm, "ci", "--omit=dev", "--no-audit", "--no-fund"], cwd=NODE_DIR,
|
|
267
318
|
label="Web adapters and code parser via npm", idle_s=300)
|
|
268
319
|
if reuse_browser_runtime:
|
|
@@ -286,10 +337,12 @@ def install(reuse_browser_runtime=None):
|
|
|
286
337
|
idle_s=180, stream=True, attempts=3)
|
|
287
338
|
_run([str(BROWSER_PYTHON), "-m", "playwright", "install", "chromium"],
|
|
288
339
|
label="Chromium browser", idle_s=180, stream=True, attempts=3)
|
|
340
|
+
if not _moli_ready():
|
|
341
|
+
_install_moli()
|
|
289
342
|
status = _check()
|
|
290
343
|
if not all(status.values()):
|
|
291
344
|
raise RuntimeError(f"Incomplete web runtime: {status}")
|
|
292
|
-
print("Smart Tool runtime installed: Node, open-websearch, TypeScript, Java, Angular, Camoufox and
|
|
345
|
+
print("Smart Tool runtime installed: Node, open-websearch, TypeScript, Java, Angular, Playwright MCP, Camoufox, Crawl4AI and Moli.")
|
|
293
346
|
|
|
294
347
|
|
|
295
348
|
def main():
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@allansantos-dev/smart-tool",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.10",
|
|
4
4
|
"description": "Local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharper tools than their built-in search. Windows.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"author": "Allan Santos",
|
package/setup_ui.py
CHANGED
|
@@ -20,6 +20,7 @@ import secrets
|
|
|
20
20
|
|
|
21
21
|
import autostart
|
|
22
22
|
import block_patterns
|
|
23
|
+
import browser_control
|
|
23
24
|
import client_hooks
|
|
24
25
|
import config
|
|
25
26
|
import daemon_launcher
|
|
@@ -168,12 +169,17 @@ def handle_local_models_save(body):
|
|
|
168
169
|
|
|
169
170
|
def handle_integration_get():
|
|
170
171
|
registered = endpoint_sync.registrations()
|
|
172
|
+
try:
|
|
173
|
+
browser, browser_error = browser_control.status(), ""
|
|
174
|
+
except ValueError as exc:
|
|
175
|
+
browser, browser_error = {}, str(exc)
|
|
171
176
|
clients = []
|
|
172
177
|
for key, info in AGENT_CLIENTS.items():
|
|
173
178
|
spec, state = client_hooks.CLIENTS[key], client_hooks.status(info)
|
|
174
179
|
clients.append({"client": key, "label": spec["label"], "mcp_url": registered.get(key), "hook": state["installed"],
|
|
175
180
|
"hook_error": state["error"], "file": spec["files"][0], "docs": spec["docs"],
|
|
176
|
-
"register_command": " ".join(endpoint_sync.register_command(key, setup_origin()))
|
|
181
|
+
"register_command": " ".join(endpoint_sync.register_command(key, setup_origin())),
|
|
182
|
+
"browser_control": browser.get(key), "browser_control_error": browser_error})
|
|
177
183
|
try:
|
|
178
184
|
mode, mode_error = config.hook_mode(), ""
|
|
179
185
|
except ValueError as exc:
|
|
@@ -206,9 +212,12 @@ def handle_integration_get():
|
|
|
206
212
|
def handle_integration_save(body):
|
|
207
213
|
"""`hook_mode` sets redirect/advise/off for every client; `register_mcp` runs the client's CLI; hook in two steps:
|
|
208
214
|
`preview` shows the exact change, `install` writes it (with backup)."""
|
|
209
|
-
if isinstance(body, dict) and body.get("action") == "
|
|
215
|
+
if isinstance(body, dict) and body.get("action") == "block_patterns":
|
|
216
|
+
ids = body.get("ids")
|
|
217
|
+
if not isinstance(ids, list) or not all(isinstance(i, str) for i in ids):
|
|
218
|
+
raise SetupError("ids must be a list of pattern ids.")
|
|
210
219
|
try:
|
|
211
|
-
block_patterns.
|
|
220
|
+
block_patterns.decide_proposals(ids, body.get("decision"))
|
|
212
221
|
except (OSError, ValueError) as exc:
|
|
213
222
|
raise SetupError(str(exc))
|
|
214
223
|
return handle_integration_get()
|
|
@@ -228,14 +237,18 @@ def handle_integration_save(body):
|
|
|
228
237
|
info = AGENT_CLIENTS[body["client"]]
|
|
229
238
|
try:
|
|
230
239
|
if body.get("action") == "register_mcp":
|
|
231
|
-
|
|
240
|
+
command = endpoint_sync.register(body["client"], setup_origin())
|
|
241
|
+
return {"command": command, "browser_control": browser_control.ensure(body["client"]),
|
|
242
|
+
**handle_integration_get()}
|
|
243
|
+
if body.get("action") == "browser_control":
|
|
244
|
+
return {"browser_control": browser_control.ensure(body["client"]), **handle_integration_get()}
|
|
232
245
|
if body.get("action") == "preview":
|
|
233
246
|
return client_hooks.preview(info)
|
|
234
247
|
if body.get("action") == "install":
|
|
235
248
|
return {**client_hooks.install(info, body.get("token"), True), **handle_integration_get()}
|
|
236
249
|
except (ValueError, RuntimeError) as exc:
|
|
237
250
|
raise SetupError(str(exc))
|
|
238
|
-
raise SetupError("Invalid action: use hook_mode, register_mcp, preview or install.")
|
|
251
|
+
raise SetupError("Invalid action: use hook_mode, register_mcp, browser_control, preview or install.")
|
|
239
252
|
|
|
240
253
|
|
|
241
254
|
def handle_providers_get():
|
|
@@ -462,6 +475,29 @@ _PAGE_TEMPLATE = """<!doctype html>
|
|
|
462
475
|
.provider-head { display: flex; justify-content: space-between; align-items: baseline; gap: 10px; margin-bottom: 8px; }
|
|
463
476
|
.provider-head strong { font: 600 .98rem var(--display); letter-spacing: .02em; }
|
|
464
477
|
.blocked { color: var(--muted); font-size: .88rem; border: 1px dashed var(--line); border-radius: 10px; padding: 14px 16px; }
|
|
478
|
+
.pattern-panel, .pattern-blocking { margin-top: 16px; border: 1px solid var(--line); border-radius: 10px; padding: 14px 16px; }
|
|
479
|
+
.pattern-head { display: flex; flex-direction: column; gap: 2px; }
|
|
480
|
+
.pattern-tools { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 10px; margin-top: 12px; }
|
|
481
|
+
.pattern-bulk { margin-top: 10px; }
|
|
482
|
+
.pattern-list { list-style: none; margin: 10px 0 0; padding: 0; border-top: 1px solid var(--line); }
|
|
483
|
+
.pattern-row { display: grid; grid-template-columns: auto minmax(0, 1fr); gap: 10px; align-items: start;
|
|
484
|
+
padding: 10px 0; border-bottom: 1px solid var(--line); }
|
|
485
|
+
.pattern-blocking .pattern-row { grid-template-columns: minmax(0, 1fr); }
|
|
486
|
+
.pattern-row input[type=checkbox] { margin-top: 6px; }
|
|
487
|
+
.pattern-body summary { display: flex; gap: 8px; align-items: baseline; cursor: pointer; min-width: 0; }
|
|
488
|
+
.pattern-tool { font: 600 .74rem var(--display); letter-spacing: .06em; text-transform: uppercase; color: var(--muted); flex: none; }
|
|
489
|
+
.pattern-code { font: .78rem/1.4 var(--mono); overflow-wrap: anywhere; min-width: 0; display: -webkit-box;
|
|
490
|
+
-webkit-line-clamp: 2; -webkit-box-orient: vertical; overflow: hidden; }
|
|
491
|
+
.pattern-body[open] .pattern-code { -webkit-line-clamp: unset; }
|
|
492
|
+
.pattern-main { min-width: 0; }
|
|
493
|
+
.pattern-meta { display: flex; justify-content: space-between; align-items: center; gap: 10px; margin-top: 6px; flex-wrap: wrap; }
|
|
494
|
+
.pattern-detail { font-size: .84rem; padding: 4px 0 2px; }
|
|
495
|
+
.pattern-detail p { margin: 4px 0; }
|
|
496
|
+
.pattern-reach { font: .78rem var(--mono); color: var(--muted); white-space: nowrap; }
|
|
497
|
+
.pattern-actions { display: flex; gap: 6px; }
|
|
498
|
+
.pattern-actions button { padding: 4px 10px; font-size: .78rem; }
|
|
499
|
+
.pattern-pager { justify-content: space-between; }
|
|
500
|
+
.pattern-blocking > summary { cursor: pointer; font: 600 .9rem var(--display); }
|
|
465
501
|
|
|
466
502
|
body:not(.ready) .station[data-live="1"] .wire::after { transform: scaleX(0); }
|
|
467
503
|
@media (max-width: 900px) { .panels { grid-template-columns: minmax(0, 1fr); } }
|
|
@@ -897,6 +933,14 @@ $("btn-save-local-models").addEventListener("click", async () => {
|
|
|
897
933
|
}
|
|
898
934
|
});
|
|
899
935
|
|
|
936
|
+
function browserText(item) {
|
|
937
|
+
if (item.browser_control_error) return "Browser control status unreadable: " + item.browser_control_error;
|
|
938
|
+
const state = item.browser_control || {};
|
|
939
|
+
if (state.state === "registered") return "Browser control on: Playwright MCP drives an isolated, headless Chromium";
|
|
940
|
+
if (state.state === "kept") return "Browser control: your own playwright MCP server is kept";
|
|
941
|
+
if (state.state === "unavailable") return "Browser control unavailable on this machine (" + state.reason + "); pages are still read by web_fetch";
|
|
942
|
+
return "Browser control is set up when the MCP is registered";
|
|
943
|
+
}
|
|
900
944
|
async function loadAgents() {
|
|
901
945
|
try {
|
|
902
946
|
const resp = await fetch("/setup/integration");
|
|
@@ -918,13 +962,28 @@ async function loadAgents() {
|
|
|
918
962
|
row.className = "provider";
|
|
919
963
|
const mcp = item.mcp_url ? "MCP registered: " + item.mcp_url : "smart-tool MCP not registered in this client";
|
|
920
964
|
const hook = item.hook_error ? "Hook unreadable: " + item.hook_error : item.hook ? "Hook installed and up to date" : "Hook missing or outdated";
|
|
921
|
-
row.innerHTML = `<div class="provider-head"><strong></strong><span class="state"></span></div><p class="hint mcp"></p><p class="hint mono cmd"></p><p class="hint file"></p>
|
|
922
|
-
<div class="actions"><button type="button" class="register">Register MCP</button><button type="button" class="hook">Install or update hook</button></div>
|
|
965
|
+
row.innerHTML = `<div class="provider-head"><strong></strong><span class="state"></span></div><p class="hint mcp"></p><p class="hint browser"></p><p class="hint mono cmd"></p><p class="hint file"></p>
|
|
966
|
+
<div class="actions"><button type="button" class="register">Register MCP</button><button type="button" class="hook">Install or update hook</button><button type="button" class="quiet browser-retry">Set up browser control</button></div>
|
|
923
967
|
<div class="plan" hidden><span class="hint plan-file"></span><pre class="code-preview"></pre>
|
|
924
968
|
<div class="actions"><button type="button" class="confirm">Confirm and write</button><button type="button" class="quiet cancel">Cancel</button></div></div>`;
|
|
925
969
|
row.querySelector("strong").textContent = item.label;
|
|
926
970
|
row.querySelector(".state").textContent = hook;
|
|
927
971
|
row.querySelector(".mcp").textContent = mcp;
|
|
972
|
+
row.querySelector(".browser").textContent = browserText(item);
|
|
973
|
+
const retry = row.querySelector(".browser-retry");
|
|
974
|
+
retry.hidden = !item.mcp_url || ["registered", "kept"].includes((item.browser_control || {}).state);
|
|
975
|
+
retry.addEventListener("click", async () => {
|
|
976
|
+
retry.disabled = true;
|
|
977
|
+
setMsg("agents-msg", "", "Checking whether agents can drive a browser here…");
|
|
978
|
+
try {
|
|
979
|
+
const result = await post("/setup/integration", { client: item.client, action: "browser_control" });
|
|
980
|
+
setMsg("agents-msg", "ok", item.label + ": " + browserText({ browser_control: result.browser_control }));
|
|
981
|
+
await loadAgents();
|
|
982
|
+
} catch (error) {
|
|
983
|
+
setMsg("agents-msg", "error", error.message);
|
|
984
|
+
retry.disabled = false;
|
|
985
|
+
}
|
|
986
|
+
});
|
|
928
987
|
row.querySelector(".file").textContent = item.file;
|
|
929
988
|
row.querySelector(".cmd").textContent = item.mcp_url ? "" : "Or in a terminal: " + item.register_command;
|
|
930
989
|
const register = row.querySelector(".register");
|
|
@@ -934,7 +993,7 @@ async function loadAgents() {
|
|
|
934
993
|
setMsg("agents-msg", "", "Registering the MCP in " + item.label + "…");
|
|
935
994
|
try {
|
|
936
995
|
const result = await post("/setup/integration", { client: item.client, action: "register_mcp" });
|
|
937
|
-
setMsg("agents-msg", "ok", "MCP registered with: " + result.command + ". Reconnect open sessions (/mcp).");
|
|
996
|
+
setMsg("agents-msg", "ok", "MCP registered with: " + result.command + ". " + browserText({ browser_control: result.browser_control }) + ". Reconnect open sessions (/mcp).");
|
|
938
997
|
await loadAgents();
|
|
939
998
|
} catch (error) {
|
|
940
999
|
setMsg("agents-msg", "error", error.message);
|
|
@@ -973,49 +1032,163 @@ async function loadAgents() {
|
|
|
973
1032
|
setMsg("agents-msg", "error", "Agent status unavailable: " + error.message);
|
|
974
1033
|
}
|
|
975
1034
|
}
|
|
1035
|
+
const PATTERN_PAGE_SIZE = 8;
|
|
1036
|
+
const patternView = { page: 0, filter: "", sort: "impact", selected: new Set(), open: false };
|
|
1037
|
+
let lastPatterns = null;
|
|
1038
|
+
function patternPage(items, view) {
|
|
1039
|
+
const needle = view.filter.trim().toLowerCase();
|
|
1040
|
+
const matched = items.filter((item) => !needle || [item.tool, item.pattern, item.reason, item.example]
|
|
1041
|
+
.some((text) => String(text || "").toLowerCase().includes(needle)));
|
|
1042
|
+
const sorted = matched.slice().sort(view.sort === "newest"
|
|
1043
|
+
? (a, b) => (b.proposed_at || 0) - (a.proposed_at || 0)
|
|
1044
|
+
: (a, b) => (b.history_matches || 0) - (a.history_matches || 0) || (b.proposed_at || 0) - (a.proposed_at || 0));
|
|
1045
|
+
const pages = Math.max(1, Math.ceil(sorted.length / PATTERN_PAGE_SIZE));
|
|
1046
|
+
const page = Math.min(Math.max(0, view.page), pages - 1);
|
|
1047
|
+
return { rows: sorted.slice(page * PATTERN_PAGE_SIZE, (page + 1) * PATTERN_PAGE_SIZE), page, pages,
|
|
1048
|
+
matched: sorted.length, total: items.length,
|
|
1049
|
+
calls: items.reduce((sum, item) => sum + (item.history_matches || 0), 0) };
|
|
1050
|
+
}
|
|
1051
|
+
function patternEl(tag, className, text) {
|
|
1052
|
+
const el = document.createElement(tag);
|
|
1053
|
+
if (className) el.className = className;
|
|
1054
|
+
if (text !== undefined) el.textContent = text;
|
|
1055
|
+
return el;
|
|
1056
|
+
}
|
|
1057
|
+
async function decidePatterns(ids, decision, label) {
|
|
1058
|
+
try {
|
|
1059
|
+
const result = await post("/setup/integration", { action: "block_patterns", ids, decision });
|
|
1060
|
+
for (const id of ids) patternView.selected.delete(id);
|
|
1061
|
+
renderPatterns(result.block_patterns);
|
|
1062
|
+
setMsg("agents-msg", "ok", label + ": " + ids.length + " pattern(s).");
|
|
1063
|
+
} catch (error) {
|
|
1064
|
+
setMsg("agents-msg", "error", error.message);
|
|
1065
|
+
}
|
|
1066
|
+
}
|
|
1067
|
+
function patternRow(item, actions, selectable) {
|
|
1068
|
+
const row = patternEl("li", "pattern-row");
|
|
1069
|
+
if (selectable) {
|
|
1070
|
+
const check = patternEl("input");
|
|
1071
|
+
check.type = "checkbox";
|
|
1072
|
+
check.checked = patternView.selected.has(item.id);
|
|
1073
|
+
check.setAttribute("aria-label", "Select " + item.pattern);
|
|
1074
|
+
check.addEventListener("change", () => {
|
|
1075
|
+
if (check.checked) patternView.selected.add(item.id); else patternView.selected.delete(item.id);
|
|
1076
|
+
renderPatterns(lastPatterns);
|
|
1077
|
+
});
|
|
1078
|
+
row.appendChild(check);
|
|
1079
|
+
}
|
|
1080
|
+
const body = patternEl("details", "pattern-body");
|
|
1081
|
+
const summary = patternEl("summary");
|
|
1082
|
+
const code = patternEl("code", "pattern-code", item.pattern);
|
|
1083
|
+
code.title = item.pattern;
|
|
1084
|
+
summary.append(patternEl("span", "pattern-tool", item.tool), code);
|
|
1085
|
+
const detail = patternEl("div", "pattern-detail");
|
|
1086
|
+
detail.append(patternEl("p", "", item.reason), patternEl("p", "hint mono", "Example: " + item.example),
|
|
1087
|
+
patternEl("p", "hint mono", item.pattern));
|
|
1088
|
+
body.append(summary, detail);
|
|
1089
|
+
const reach = patternEl("span", "pattern-reach", item.history_total ? item.history_matches + " / " + item.history_total : "–");
|
|
1090
|
+
reach.title = item.history_total ? "Would have blocked " + item.history_matches + " of " + item.history_total + " logged " + item.tool + " calls" : "No logged calls to compare";
|
|
1091
|
+
const buttons = patternEl("span", "pattern-actions");
|
|
1092
|
+
for (const [decision, label] of actions) {
|
|
1093
|
+
const button = patternEl("button", "quiet", label);
|
|
1094
|
+
button.type = "button";
|
|
1095
|
+
button.addEventListener("click", () => decidePatterns([item.id], decision, label));
|
|
1096
|
+
buttons.appendChild(button);
|
|
1097
|
+
}
|
|
1098
|
+
const meta = patternEl("div", "pattern-meta");
|
|
1099
|
+
meta.append(reach, buttons);
|
|
1100
|
+
const main = patternEl("div", "pattern-main");
|
|
1101
|
+
main.append(body, meta);
|
|
1102
|
+
row.appendChild(main);
|
|
1103
|
+
return row;
|
|
1104
|
+
}
|
|
976
1105
|
function renderPatterns(patterns) {
|
|
1106
|
+
lastPatterns = patterns;
|
|
977
1107
|
const box = $("patterns-list");
|
|
978
1108
|
box.textContent = "";
|
|
979
|
-
if (patterns.error) {
|
|
980
|
-
const
|
|
981
|
-
|
|
982
|
-
|
|
983
|
-
|
|
984
|
-
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
989
|
-
|
|
990
|
-
|
|
991
|
-
|
|
992
|
-
|
|
993
|
-
|
|
994
|
-
|
|
995
|
-
|
|
996
|
-
|
|
997
|
-
|
|
998
|
-
|
|
999
|
-
|
|
1000
|
-
|
|
1001
|
-
|
|
1002
|
-
|
|
1003
|
-
|
|
1004
|
-
|
|
1005
|
-
|
|
1006
|
-
|
|
1007
|
-
|
|
1008
|
-
|
|
1009
|
-
setMsg("agents-msg", "error", error.message);
|
|
1010
|
-
}
|
|
1011
|
-
});
|
|
1012
|
-
buttons.appendChild(button);
|
|
1013
|
-
}
|
|
1014
|
-
row.appendChild(buttons);
|
|
1015
|
-
box.appendChild(row);
|
|
1109
|
+
if (patterns.error) { box.appendChild(patternEl("p", "msg error", patterns.error)); return; }
|
|
1110
|
+
const proposals = patterns.proposals;
|
|
1111
|
+
const live = new Set(proposals.map((item) => item.id));
|
|
1112
|
+
for (const id of [...patternView.selected]) if (!live.has(id)) patternView.selected.delete(id);
|
|
1113
|
+
if (proposals.length) {
|
|
1114
|
+
const view = patternPage(proposals, patternView);
|
|
1115
|
+
patternView.page = view.page;
|
|
1116
|
+
const panel = patternEl("div", "pattern-panel");
|
|
1117
|
+
const head = patternEl("div", "pattern-head");
|
|
1118
|
+
head.append(patternEl("strong", "", proposals.length + " proposed pattern(s) waiting"),
|
|
1119
|
+
patternEl("span", "hint", "Together they would have blocked " + view.calls + " logged call(s). Open a row for its reason and example."));
|
|
1120
|
+
const tools = patternEl("div", "pattern-tools");
|
|
1121
|
+
const filter = patternEl("input");
|
|
1122
|
+
filter.type = "text";
|
|
1123
|
+
filter.placeholder = "Filter by pattern, reason or example";
|
|
1124
|
+
filter.value = patternView.filter;
|
|
1125
|
+
filter.setAttribute("aria-label", "Filter proposed patterns");
|
|
1126
|
+
filter.addEventListener("input", () => {
|
|
1127
|
+
patternView.filter = filter.value;
|
|
1128
|
+
patternView.page = 0;
|
|
1129
|
+
renderPatterns(lastPatterns);
|
|
1130
|
+
const again = document.querySelector(".pattern-tools input[type=text]");
|
|
1131
|
+
if (again) { again.focus(); again.setSelectionRange(again.value.length, again.value.length); }
|
|
1132
|
+
});
|
|
1133
|
+
const sort = patternEl("select");
|
|
1134
|
+
sort.setAttribute("aria-label", "Order of proposed patterns");
|
|
1135
|
+
for (const [value, label] of [["impact", "Most blocked calls first"], ["newest", "Newest first"]]) {
|
|
1136
|
+
const option = patternEl("option", "", label);
|
|
1137
|
+
option.value = value;
|
|
1138
|
+
sort.appendChild(option);
|
|
1016
1139
|
}
|
|
1140
|
+
sort.value = patternView.sort;
|
|
1141
|
+
sort.addEventListener("change", () => { patternView.sort = sort.value; patternView.page = 0; renderPatterns(lastPatterns); });
|
|
1142
|
+
tools.append(filter, sort);
|
|
1143
|
+
const bulk = patternEl("div", "actions pattern-bulk");
|
|
1144
|
+
const pageIds = view.rows.map((item) => item.id);
|
|
1145
|
+
const allOnPage = pageIds.length > 0 && pageIds.every((id) => patternView.selected.has(id));
|
|
1146
|
+
const selectPage = patternEl("button", "quiet", allOnPage ? "Clear this page" : "Select this page");
|
|
1147
|
+
selectPage.type = "button";
|
|
1148
|
+
selectPage.disabled = !pageIds.length;
|
|
1149
|
+
selectPage.addEventListener("click", () => {
|
|
1150
|
+
for (const id of pageIds) if (allOnPage) patternView.selected.delete(id); else patternView.selected.add(id);
|
|
1151
|
+
renderPatterns(lastPatterns);
|
|
1152
|
+
});
|
|
1153
|
+
const count = patternView.selected.size;
|
|
1154
|
+
const accept = patternEl("button", "", "Accept selected (" + count + ")");
|
|
1155
|
+
const reject = patternEl("button", "quiet", "Reject selected (" + count + ")");
|
|
1156
|
+
for (const [button, decision, label] of [[accept, "accept", "Accepted"], [reject, "reject", "Rejected"]]) {
|
|
1157
|
+
button.type = "button";
|
|
1158
|
+
button.disabled = !count;
|
|
1159
|
+
button.addEventListener("click", () => decidePatterns([...patternView.selected], decision, label));
|
|
1160
|
+
}
|
|
1161
|
+
bulk.append(selectPage, accept, reject);
|
|
1162
|
+
const list = patternEl("ul", "pattern-list");
|
|
1163
|
+
for (const item of view.rows) list.appendChild(patternRow(item, [["accept", "Accept"], ["reject", "Reject"]], true));
|
|
1164
|
+
if (!view.rows.length) list.appendChild(patternEl("li", "hint", "No proposal matches the filter."));
|
|
1165
|
+
const pager = patternEl("div", "actions pattern-pager");
|
|
1166
|
+
const prev = patternEl("button", "quiet", "Previous");
|
|
1167
|
+
const next = patternEl("button", "quiet", "Next");
|
|
1168
|
+
prev.type = "button";
|
|
1169
|
+
next.type = "button";
|
|
1170
|
+
prev.disabled = view.page === 0;
|
|
1171
|
+
next.disabled = view.page >= view.pages - 1;
|
|
1172
|
+
prev.addEventListener("click", () => { patternView.page -= 1; renderPatterns(lastPatterns); });
|
|
1173
|
+
next.addEventListener("click", () => { patternView.page += 1; renderPatterns(lastPatterns); });
|
|
1174
|
+
const first = view.page * PATTERN_PAGE_SIZE;
|
|
1175
|
+
const shown = view.matched ? (first + 1) + "–" + (first + view.rows.length) : "0";
|
|
1176
|
+
const filtered = view.matched < view.total ? " (filtered from " + view.total + ")" : "";
|
|
1177
|
+
pager.append(prev, patternEl("span", "hint", shown + " of " + view.matched + filtered + " · page " + (view.page + 1) + " of " + view.pages), next);
|
|
1178
|
+
panel.append(head, tools, bulk, list, pager);
|
|
1179
|
+
box.appendChild(panel);
|
|
1180
|
+
}
|
|
1181
|
+
if (patterns.accepted.length) {
|
|
1182
|
+
const blocking = patternEl("details", "pattern-blocking");
|
|
1183
|
+
blocking.open = patternView.open;
|
|
1184
|
+
blocking.addEventListener("toggle", () => { patternView.open = blocking.open; });
|
|
1185
|
+
blocking.appendChild(patternEl("summary", "", "Blocking patterns (" + patterns.accepted.length + ")"));
|
|
1186
|
+
const list = patternEl("ul", "pattern-list");
|
|
1187
|
+
for (const item of patterns.accepted) list.appendChild(patternRow(item, [["remove", "Remove"]], false));
|
|
1188
|
+
blocking.appendChild(list);
|
|
1189
|
+
box.appendChild(blocking);
|
|
1017
1190
|
}
|
|
1018
|
-
if (patterns.rejected)
|
|
1191
|
+
if (patterns.rejected) box.appendChild(patternEl("p", "hint", patterns.rejected + " rejected proposal(s) are not offered again."));
|
|
1019
1192
|
}
|
|
1020
1193
|
loadAgents();
|
|
1021
1194
|
$("btn-save-pattern-proposals").addEventListener("click", async () => {
|
package/smart_tool_daemon.py
CHANGED
|
@@ -37,6 +37,7 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|
|
37
37
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
|
38
38
|
import capabilities
|
|
39
39
|
import client_hooks
|
|
40
|
+
import install_runtime
|
|
40
41
|
import integration_metrics
|
|
41
42
|
import config
|
|
42
43
|
import freshness
|
|
@@ -3187,17 +3188,52 @@ def _tool_result(text, is_error=False):
|
|
|
3187
3188
|
|
|
3188
3189
|
WEB_FETCH_TIMEOUT_S = 90
|
|
3189
3190
|
WEB_FETCH_RENDER_TIMEOUT_S = 60
|
|
3191
|
+
MOLI_RENDER_TIMEOUT_S = 25
|
|
3192
|
+
# Engine of the last render on this request's thread: web_fetch.read_page calls _render_page on the handler's thread.
|
|
3193
|
+
_RENDER = threading.local()
|
|
3190
3194
|
WEB_FETCH_VECTOR_ROOT = os.path.join(paths.DATA_DIR, "web-fetch")
|
|
3191
3195
|
|
|
3192
3196
|
|
|
3197
|
+
def _moli_markdown(url, timeout):
|
|
3198
|
+
"""Markdown of a JavaScript page rendered by Moli (layout and paint only when the page needs them)."""
|
|
3199
|
+
if not install_runtime.MOLI_EXE.is_file():
|
|
3200
|
+
raise RuntimeError(f"Moli is not installed ({install_runtime.MOLI_EXE}); reinstall Smart Tool.")
|
|
3201
|
+
done = subprocess.run([str(install_runtime.MOLI_EXE), "fetch", "--dump", "markdown", url], capture_output=True,
|
|
3202
|
+
timeout=timeout, creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0))
|
|
3203
|
+
if done.returncode:
|
|
3204
|
+
raise RuntimeError(f"moli exited {done.returncode}: {done.stderr.decode('utf-8', 'replace').strip()[-300:]}")
|
|
3205
|
+
return done.stdout.decode("utf-8", "replace")
|
|
3206
|
+
|
|
3207
|
+
|
|
3193
3208
|
def _render_page(url, deadline):
|
|
3209
|
+
"""Moli first; Chromium through Crawl4AI only when Moli fails or comes back thin. Measured on 2026-10-10 over 21
|
|
3210
|
+
pages HTTP could not read: Moli 1.6 s and 90 MB peak (median), Chromium 4.2 s and 581 MB; Moli read 14, Chromium
|
|
3211
|
+
16, both together 17 (only Chromium read YouTube, Google Maps and Airbnb). The engine that answered goes to the
|
|
3212
|
+
web_fetch metric through _RENDER."""
|
|
3194
3213
|
left = deadline - time.monotonic()
|
|
3195
3214
|
if left < 5:
|
|
3196
3215
|
return ""
|
|
3216
|
+
_RENDER.engine, _RENDER.moli_error = None, None
|
|
3217
|
+
try:
|
|
3218
|
+
markdown = _moli_markdown(url, min(MOLI_RENDER_TIMEOUT_S, left - 2))
|
|
3219
|
+
except (OSError, RuntimeError, subprocess.TimeoutExpired) as exc:
|
|
3220
|
+
markdown, _RENDER.moli_error = "", f"{type(exc).__name__}: {exc}"[:200]
|
|
3221
|
+
_log(f"web_fetch Moli render failed, trying Chromium: {_RENDER.moli_error}")
|
|
3222
|
+
if not web_fetch.thin(markdown):
|
|
3223
|
+
_RENDER.engine = "moli"
|
|
3224
|
+
return markdown
|
|
3225
|
+
left = deadline - time.monotonic()
|
|
3226
|
+
if left < 5:
|
|
3227
|
+
return markdown
|
|
3197
3228
|
raw = _run_search_script(CAMOUFOX_VENV_PYTHON, CRAWL4AI_SEARCH_SCRIPT, [url, "--max-pages", "1"],
|
|
3198
3229
|
min(WEB_FETCH_RENDER_TIMEOUT_S, left))
|
|
3199
3230
|
pages = raw.get("pages") if isinstance(raw, dict) else None
|
|
3200
|
-
|
|
3231
|
+
chromium = (pages[0].get("markdown") or "") if pages else ""
|
|
3232
|
+
if len(chromium.strip()) > len(markdown.strip()):
|
|
3233
|
+
_RENDER.engine = "chromium"
|
|
3234
|
+
return chromium
|
|
3235
|
+
_RENDER.engine = "moli"
|
|
3236
|
+
return markdown
|
|
3201
3237
|
|
|
3202
3238
|
|
|
3203
3239
|
def _web_fetch_embed(texts, deadline):
|
|
@@ -3243,6 +3279,7 @@ def _handle_web_fetch(arguments):
|
|
|
3243
3279
|
if problem:
|
|
3244
3280
|
raise RuntimeError(f"Gateway model unavailable ({problem}).")
|
|
3245
3281
|
deadline = started + WEB_FETCH_TIMEOUT_S
|
|
3282
|
+
_RENDER.engine, _RENDER.moli_error = None, None
|
|
3246
3283
|
page = web_fetch.read_page(url, deadline, _render_page)
|
|
3247
3284
|
selected, pieces = web_fetch.select(page["text"], prompt, lambda texts: _web_fetch_embed(texts, deadline))
|
|
3248
3285
|
text = web_fetch.answer(config.load_config()["router_model"], prompt, {**page, "text": selected}, deadline)
|
|
@@ -3252,6 +3289,7 @@ def _handle_web_fetch(arguments):
|
|
|
3252
3289
|
raise RuntimeError(f"web_fetch failed: {str(exc).rstrip('.')}. The native WebFetch is allowed for this URL for "
|
|
3253
3290
|
f"{web_fetch.FAILED_TTL_S // 60} min.") from None
|
|
3254
3291
|
_metric("web_fetch", cached=page["cached"], rendered=page["rendered"], render_error=page.get("render_error"),
|
|
3292
|
+
render_engine=_RENDER.engine if page["rendered"] else None, moli_error=_RENDER.moli_error,
|
|
3255
3293
|
page_chars=page["chars"], read_chars=len(selected), pieces=pieces,
|
|
3256
3294
|
answer_chars=len(text), elapsed_s=round(time.monotonic() - started, 2))
|
|
3257
3295
|
origin = "from cache" if page["cached"] else "rendered in the browser" if page["rendered"] else "downloaded now"
|
package/version.py
CHANGED
|
@@ -1,14 +1,15 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "smart-tool-web-adapters",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "0.9.0",
|
|
4
4
|
"lockfileVersion": 3,
|
|
5
5
|
"requires": true,
|
|
6
6
|
"packages": {
|
|
7
7
|
"": {
|
|
8
8
|
"name": "smart-tool-web-adapters",
|
|
9
|
-
"version": "
|
|
9
|
+
"version": "0.9.0",
|
|
10
10
|
"dependencies": {
|
|
11
11
|
"@angular/compiler": "20.3.0",
|
|
12
|
+
"@playwright/mcp": "0.0.83",
|
|
12
13
|
"java-parser": "3.0.1",
|
|
13
14
|
"open-websearch": "2.1.11",
|
|
14
15
|
"typescript": "5.9.3"
|
|
@@ -588,6 +589,22 @@
|
|
|
588
589
|
"node": ">=14.0.0"
|
|
589
590
|
}
|
|
590
591
|
},
|
|
592
|
+
"node_modules/@playwright/mcp": {
|
|
593
|
+
"version": "0.0.83",
|
|
594
|
+
"resolved": "https://registry.npmjs.org/@playwright/mcp/-/mcp-0.0.83.tgz",
|
|
595
|
+
"integrity": "sha512-oNcl+Ae2/IAjhfPeP46BfIkSakfmprY+aOtkv5MjrQ4lPav4/yNtPhL0iq8SlIM90oApWgBDUxaNKvktazUKOg==",
|
|
596
|
+
"license": "Apache-2.0",
|
|
597
|
+
"dependencies": {
|
|
598
|
+
"playwright": "1.64.0-alpha-1790635538000",
|
|
599
|
+
"playwright-core": "1.64.0-alpha-1790635538000"
|
|
600
|
+
},
|
|
601
|
+
"bin": {
|
|
602
|
+
"playwright-mcp": "cli.js"
|
|
603
|
+
},
|
|
604
|
+
"engines": {
|
|
605
|
+
"node": ">=18"
|
|
606
|
+
}
|
|
607
|
+
},
|
|
591
608
|
"node_modules/@types/cheerio": {
|
|
592
609
|
"version": "0.22.35",
|
|
593
610
|
"resolved": "https://registry.npmjs.org/@types/cheerio/-/cheerio-0.22.35.tgz",
|
|
@@ -2087,6 +2104,33 @@
|
|
|
2087
2104
|
"node": ">=16.20.0"
|
|
2088
2105
|
}
|
|
2089
2106
|
},
|
|
2107
|
+
"node_modules/playwright": {
|
|
2108
|
+
"version": "1.64.0-alpha-1790635538000",
|
|
2109
|
+
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.64.0-alpha-1790635538000.tgz",
|
|
2110
|
+
"integrity": "sha512-/5XDUMxpOd/9AojJtWDFIRV3EJZNQ0AwvK69wgt3CPzy8wTgiBG9pX45VfxtqtvIRGwfWzMhrvawNKreG69Snw==",
|
|
2111
|
+
"license": "Apache-2.0",
|
|
2112
|
+
"dependencies": {
|
|
2113
|
+
"playwright-core": "1.64.0-alpha-1790635538000"
|
|
2114
|
+
},
|
|
2115
|
+
"bin": {
|
|
2116
|
+
"playwright": "cli.js"
|
|
2117
|
+
},
|
|
2118
|
+
"engines": {
|
|
2119
|
+
"node": ">=20"
|
|
2120
|
+
}
|
|
2121
|
+
},
|
|
2122
|
+
"node_modules/playwright-core": {
|
|
2123
|
+
"version": "1.64.0-alpha-1790635538000",
|
|
2124
|
+
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.64.0-alpha-1790635538000.tgz",
|
|
2125
|
+
"integrity": "sha512-pNwaXirhXMRLaRQs4NQ18EpTdtDoFwyPH9FOaaDC7YXG4hpwE2xmwCic/1srbuExVQC7zT9LJSJpDBAru+Vo9A==",
|
|
2126
|
+
"license": "Apache-2.0",
|
|
2127
|
+
"bin": {
|
|
2128
|
+
"playwright-core": "cli.js"
|
|
2129
|
+
},
|
|
2130
|
+
"engines": {
|
|
2131
|
+
"node": ">=20"
|
|
2132
|
+
}
|
|
2133
|
+
},
|
|
2090
2134
|
"node_modules/proxy-addr": {
|
|
2091
2135
|
"version": "2.0.8",
|
|
2092
2136
|
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.8.tgz",
|