@allansantos-dev/smart-tool 0.9.8 → 0.9.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,31 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.9.10 - beta
4
+
5
+ - Browser control for agents, on by default: registering smart-tool in Claude Code or Codex (and every update, for
6
+ clients where it is registered) also registers Microsoft's Playwright MCP (`@playwright/mcp` 0.0.83, pinned in
7
+ `web_adapters/node`) as the `playwright` server, driving the Chromium Smart Tool already installs, headless, with an
8
+ isolated in-memory profile. A probe first starts the server and opens a blank page the way an agent would; where
9
+ that fails (a corporate policy against remote debugging, a missing browser, the client CLI refusing) nothing is
10
+ registered, agents keep the read-only browsers of `web_fetch` and `web_search`, and the setup screen shows the
11
+ reason and a retry button instead of an error. A registration Smart Tool made is refreshed on update (Node and
12
+ Chromium paths change) and removed when the probe stops passing; a `playwright` server you registered is kept.
13
+ - Block pattern proposals no longer stack one block per proposal (21 pending ones made the setup screen scroll
14
+ forever): one approval panel shows 8 per page, ordered by how many logged calls each would have blocked, with a
15
+ filter, a select-page toggle and bulk accept/reject in a single write (all ids or none); reason and example open on
16
+ demand, and blocking patterns sit in a collapsed list.
17
+
18
+ ## 0.9.9 - beta
19
+
20
+ - `web_fetch` renders JavaScript pages with [Moli](https://github.com/lexmount/moli) first (a headless browser built
21
+ for agents: layout and paint only when a page needs them; 43 MB download, SHA-256 checked) and starts Chromium
22
+ through Crawl4AI only when Moli fails or comes back thin. On 21 pages HTTP could not read: Moli 1.6 s and 90 MB
23
+ peak (median) against 4.2 s and 581 MB; Moli read 14, Chromium 16, both together 17 (only Chromium read YouTube,
24
+ Google Maps and Airbnb). With several agent sessions reading at once, measured on N simultaneous rendered reads:
25
+ aggregate peak memory 448 → 35 MB (1), 1,788 → 194 MB (3), 2,901 → 634 MB (5), 4.4-4.7 GB → 1.0-1.4 GB (10), and
26
+ the median read time halved (10 at once: 5.0 s → 2.5 s). Pages only Chromium reads now take Moli's 1-4 s more.
27
+ The `web_fetch` metric records which engine read the page and Moli's error when it fell back.
28
+
3
29
  ## 0.9.8 - beta
4
30
 
5
31
  - An update could keep running the previous code: npm dates every file in the package 1985-10-26, the installer kept
package/NOTICE CHANGED
@@ -15,6 +15,7 @@ Python packages (installation venv, from PyPI)
15
15
 
16
16
  Browser runtime (web_adapters/browser/.venv, from PyPI and the projects' download servers)
17
17
  - camoufox Python package (MIT) and the Camoufox browser, a Firefox fork (MPL-2.0), fetched by `camoufox fetch`
18
+ - Moli (Apache-2.0 OR MIT), https://github.com/lexmount/moli: renders JavaScript pages for web_fetch; downloaded at install time from its GitHub release (SHA-256 checked), with its own license files
18
19
  - crawl4ai (Apache-2.0), https://github.com/unclecode/crawl4ai: page rendering is powered by Crawl4AI
19
20
  - Playwright (Apache-2.0) and the Chromium build it installs (BSD-style, see https://www.chromium.org)
20
21
 
@@ -24,6 +25,8 @@ Node runtime and packages (web_adapters/node, from nodejs.org and npm)
24
25
  - typescript (Apache-2.0)
25
26
  - java-parser (Apache-2.0)
26
27
  - @angular/compiler (MIT)
28
+ - @playwright/mcp (Apache-2.0), https://github.com/microsoft/playwright-mcp: browser control for agents, registered as
29
+ the `playwright` MCP server over the Chromium above
27
30
  - their dependencies as pinned in web_adapters/node/package-lock.json (MIT, ISC, BSD and Apache-2.0)
28
31
 
29
32
  Web search sources
package/README.md CHANGED
@@ -9,6 +9,9 @@ A local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharpe
9
9
  | `web_fetch` | Reads a known URL and returns only the answer to your prompt, written by a small model; pages are cached for 24 h. Pages over 60,000 characters are not cut: the model reads the parts closest to the prompt (needs an `embedding_model`) |
10
10
  | `project_manage` | Registers and indexes projects, shows coverage, code maps, duplicated functions and index storage; `graph` with `symbol` tells what changing a function touches: callers, calls, the tests that reach it and the files that import it |
11
11
 
12
+ Agents that must click, type or log in also get Microsoft's [Playwright MCP](https://github.com/microsoft/playwright-mcp),
13
+ registered as the `playwright` server where this machine lets a browser be driven (see Set up, step 3).
14
+
12
15
  An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
13
16
  are cheaper, or just tells the agent what Smart Tool would do.
14
17
 
@@ -51,7 +54,18 @@ Open the setup screen from the tray icon ("Open settings") or at `http://127.0.0
51
54
  reasoning: web search tier order and result checks), the index classification model (also reviews block patterns) and the research model. Any id the gateway
52
55
  accepts works; the catalog is shown as suggestions.
53
56
  3. **Agents**: register the MCP server in Claude Code or Codex (runs their official `mcp add` command), install the
54
- hook and choose its behavior:
57
+ hook and choose its behavior.
58
+
59
+ **Browser control** comes with the registration, and with every update for clients where smart-tool is registered:
60
+ the official Playwright MCP (pinned in `web_adapters/node`) is registered as the `playwright` server, driving the
61
+ Chromium Smart Tool installs, headless, with an isolated in-memory profile (no logins of yours) and its files in
62
+ `~/.smart-tool/data/playwright-mcp`. Before registering, Smart Tool starts that server and opens a blank page as an
63
+ agent would; where that fails (a corporate policy that forbids remote debugging, a missing browser) nothing is
64
+ registered, agents keep reading pages through `web_fetch`, and the setup screen shows the reason with a retry
65
+ button. A `playwright` server you registered yourself is never touched. Remove it with
66
+ `claude mcp remove playwright -s user` / `codex mcp remove playwright`.
67
+
68
+ The hook behaviors:
55
69
  - **Redirect** (default): the native tool is denied with the reason and the Smart Tool tool to use. The decision
56
70
  is a fixed rule, under a millisecond and the same every time: a content search over a project folder (`grep -r`,
57
71
  `rg`, `git grep`, the Grep tool on a folder) or over more than 20 listed files goes to `smart_search`; reading or
@@ -73,6 +87,10 @@ Open the setup screen from the tray icon ("Open settings") or at `http://127.0.0
73
87
  A documented function edited only in its body is not mentioned. `project_manage` with `action=docs` lists the
74
88
  public functions still without a docstring, for documenting a project that started without them.
75
89
 
90
+ **Block patterns waiting for you** are shown in one approval panel: 8 per page, the ones that would have blocked
91
+ the most logged calls first, a filter over pattern, reason and example, and bulk accept or reject of the selected
92
+ rows; each row opens to its reason and example.
93
+
76
94
  **Duplicate functions on edit** compares the functions an edit writes with the indexed code. **Warn** (default)
77
95
  tells the agent when a written function has the same body as an existing one (comments, spacing and docstrings
78
96
  ignored) or a near-identical one (only local names, strings and numbers changed), with the path and lines of the
@@ -146,4 +164,4 @@ user` / `codex mcp remove smart-tool`, then delete `%LOCALAPPDATA%\Programs\Smar
146
164
 
147
165
  Apache License 2.0. See `LICENSE` and `NOTICE` for third-party components.
148
166
 
149
- Page rendering is powered by [Crawl4AI](https://github.com/unclecode/crawl4ai) (Apache-2.0).
167
+ Page rendering is powered by [Moli](https://github.com/lexmount/moli) (Apache-2.0 / MIT), with [Crawl4AI](https://github.com/unclecode/crawl4ai) (Apache-2.0) as its fallback.
package/block_patterns.py CHANGED
@@ -247,20 +247,27 @@ def review_later(tool_name, tool_input, rule_reason, metrics_path, redact, log):
247
247
  return True
248
248
 
249
249
 
250
- def decide_proposal(entry_id, action):
251
- """accept moves a proposal to accepted; reject moves it to rejected; remove deletes an accepted pattern."""
250
+ def decide_proposals(entry_ids, action):
251
+ """accept moves proposals to accepted; reject moves them to rejected; remove deletes accepted patterns. All ids are
252
+ decided in one write, or none when any id is unknown (ValueError naming it)."""
253
+ if action not in ("accept", "reject", "remove"):
254
+ raise ValueError("Invalid action: use accept, reject or remove.")
255
+ wanted = list(dict.fromkeys(entry_ids))
256
+ if not wanted:
257
+ raise ValueError("No pattern selected.")
252
258
  with _LOCK:
253
259
  data = load()
254
260
  source = "accepted" if action == "remove" else "proposals"
255
- entry = next((e for e in data[source] if e["id"] == entry_id), None)
256
- if entry is None:
257
- raise ValueError(f"No {source[:-1]} with id {entry_id}.")
258
- data[source] = [e for e in data[source] if e["id"] != entry_id]
259
- if action == "accept":
260
- data["accepted"].append({**entry, "accepted_at": time.time()})
261
- elif action == "reject":
262
- data["rejected"].append({**entry, "rejected_by": "user"})
263
- elif action != "remove":
264
- raise ValueError("Invalid action: use accept, reject or remove.")
261
+ found = {e["id"]: e for e in data[source] if e["id"] in wanted}
262
+ missing = [entry_id for entry_id in wanted if entry_id not in found]
263
+ if missing:
264
+ raise ValueError(f"No {source[:-1]} with id {', '.join(missing)}.")
265
+ data[source] = [e for e in data[source] if e["id"] not in found]
266
+ now = time.time()
267
+ for entry_id in wanted:
268
+ if action == "accept":
269
+ data["accepted"].append({**found[entry_id], "accepted_at": now})
270
+ elif action == "reject":
271
+ data["rejected"].append({**found[entry_id], "rejected_by": "user"})
265
272
  _save(data)
266
273
  return data
@@ -0,0 +1,202 @@
1
+ """Browser control for agents: the official Playwright MCP server, registered by default next to smart-tool.
2
+
3
+ Smart Tool's own browsers only read pages (web_fetch, web_search). Agents that must click, type or log in get
4
+ Microsoft's Playwright MCP (pinned in web_adapters/node), driving the Chromium Smart Tool already installs, in an
5
+ isolated in-memory profile, headless. Before registering, a probe launches that browser through the server exactly
6
+ as the agent would: machines whose policy forbids it (corporate IT blocking remote debugging, a missing browser)
7
+ keep today's read-only behavior, and the reason is shown in the setup screen instead of an error. A server named
8
+ `playwright` the user already registered is left untouched.
9
+ """
10
+ import json
11
+ import os
12
+ import queue
13
+ import subprocess
14
+ import tempfile
15
+ import threading
16
+ import time
17
+
18
+ import atomic_io
19
+ import endpoint_sync
20
+ import install_runtime
21
+ import paths
22
+
23
+ SERVER = "playwright"
24
+ CLI = install_runtime.PLAYWRIGHT_MCP_CLI
25
+ STATUS_PATH = os.path.join(paths.DATA_DIR, "browser-control.json")
26
+ OUTPUT_DIR = os.path.join(paths.DATA_DIR, "playwright-mcp")
27
+ OUTPUT_MAX_BYTES = 100 * 1024 * 1024
28
+ PROBE_TIMEOUT_S = 60
29
+ _CHROMIUM_PROBE = ("from playwright.sync_api import sync_playwright\n"
30
+ "with sync_playwright() as p: print(p.chromium.executable_path)")
31
+
32
+
33
+ def chromium_executable():
34
+ """The Chromium that `playwright install chromium` put in place for the browser venv; RuntimeError when absent."""
35
+ if not install_runtime.BROWSER_PYTHON.is_file():
36
+ raise RuntimeError(f"Browser runtime missing ({install_runtime.BROWSER_PYTHON}).")
37
+ done = subprocess.run([str(install_runtime.BROWSER_PYTHON), "-c", _CHROMIUM_PROBE], capture_output=True,
38
+ text=True, encoding="utf-8", errors="replace", timeout=60,
39
+ creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0))
40
+ path = (done.stdout.strip().splitlines() or [""])[-1]
41
+ if done.returncode or not os.path.isfile(path):
42
+ raise RuntimeError(f"Chromium not found: {(done.stderr or done.stdout).strip()[-300:] or path}")
43
+ return path
44
+
45
+
46
+ def server_command():
47
+ """Absolute command line of the Playwright MCP server, as registered in the clients."""
48
+ node, _npm = install_runtime._node_commands()
49
+ if not node:
50
+ raise RuntimeError("Supported Node.js not found for Playwright MCP.")
51
+ if not CLI.is_file():
52
+ raise RuntimeError(f"Playwright MCP is not installed ({CLI}); reinstall Smart Tool.")
53
+ return [node, str(CLI), "--isolated", "--headless", "--executable-path", chromium_executable(),
54
+ "--output-dir", OUTPUT_DIR, "--output-max-size", str(OUTPUT_MAX_BYTES)]
55
+
56
+
57
+ def probe(command, timeout=PROBE_TIMEOUT_S):
58
+ """Starts the server over stdio, opens about:blank and closes it, as an agent's first call would.
59
+ Returns None when the browser opened, else the reason it did not (a policy block, a crash, a timeout)."""
60
+ with tempfile.TemporaryDirectory(prefix="st-browser-probe-", ignore_cleanup_errors=True) as workdir:
61
+ return _probe(command, workdir, timeout)
62
+
63
+
64
+ def _probe(command, workdir, timeout):
65
+ """probe() inside workdir, the server's working folder."""
66
+ flags = getattr(subprocess, "CREATE_NO_WINDOW", 0)
67
+ try:
68
+ proc = subprocess.Popen(command, cwd=workdir, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
69
+ stderr=subprocess.PIPE, text=True, encoding="utf-8", errors="replace",
70
+ creationflags=flags)
71
+ except OSError as exc:
72
+ return f"server did not start: {exc}"
73
+ lines, errors = queue.Queue(), []
74
+ threading.Thread(target=lambda: [lines.put(line) for line in proc.stdout] + [lines.put(None)], daemon=True).start()
75
+ threading.Thread(target=lambda: errors.extend(proc.stderr), daemon=True).start()
76
+ deadline = time.monotonic() + timeout
77
+
78
+ def call(request_id, method, params):
79
+ proc.stdin.write(json.dumps({"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}) + "\n")
80
+ proc.stdin.flush()
81
+ while True:
82
+ try:
83
+ line = lines.get(timeout=max(0.1, deadline - time.monotonic()))
84
+ except queue.Empty:
85
+ raise TimeoutError(f"no answer to {method} in {timeout}s") from None
86
+ if line is None:
87
+ raise RuntimeError(f"server exited: {''.join(errors).strip()[-300:] or 'no output'}")
88
+ try:
89
+ message = json.loads(line)
90
+ except ValueError:
91
+ continue
92
+ if message.get("id") == request_id:
93
+ if "error" in message:
94
+ raise RuntimeError(str(message["error"].get("message") or message["error"])[:300])
95
+ return message["result"]
96
+
97
+ try:
98
+ call(1, "initialize", {"protocolVersion": "2025-06-18", "capabilities": {},
99
+ "clientInfo": {"name": "smart-tool-probe", "version": "1"}})
100
+ proc.stdin.write(json.dumps({"jsonrpc": "2.0", "method": "notifications/initialized"}) + "\n")
101
+ result = call(2, "tools/call", {"name": "browser_navigate", "arguments": {"url": "about:blank"}})
102
+ if result.get("isError"):
103
+ text = " ".join(part.get("text", "") for part in result.get("content") or [])
104
+ return f"browser did not open: {' '.join(text.split())[:300]}"
105
+ call(3, "tools/call", {"name": "browser_close", "arguments": {}})
106
+ return None
107
+ except (OSError, RuntimeError, TimeoutError) as exc:
108
+ return str(exc)
109
+ finally:
110
+ if os.name == "nt":
111
+ subprocess.run(["taskkill", "/PID", str(proc.pid), "/T", "/F"], capture_output=True, creationflags=flags)
112
+ else:
113
+ proc.kill()
114
+ proc.wait(timeout=10)
115
+ for stream in (proc.stdin, proc.stdout, proc.stderr):
116
+ try:
117
+ stream.close()
118
+ except OSError:
119
+ pass
120
+
121
+
122
+ def registered():
123
+ """Whether each client already has a `playwright` MCP server (registered by Smart Tool or by the user)."""
124
+ found = {"claude": False, "codex": False}
125
+ try:
126
+ with open(endpoint_sync.claude_json(), encoding="utf-8") as stream:
127
+ found["claude"] = SERVER in (json.load(stream).get("mcpServers") or {})
128
+ except (OSError, ValueError, AttributeError):
129
+ pass
130
+ try:
131
+ with open(os.path.join(os.path.expanduser("~"), ".codex", "config.toml"), encoding="utf-8") as stream:
132
+ found["codex"] = f"[mcp_servers.{SERVER}]" in stream.read()
133
+ except OSError:
134
+ pass
135
+ return found
136
+
137
+
138
+ def register_command(client, command):
139
+ """Official CLI command that registers a stdio server (Claude Code: user scope, every project)."""
140
+ if client == "claude":
141
+ return ["claude", "mcp", "add", "--scope", "user", SERVER, "--", *command]
142
+ if client == "codex":
143
+ return ["codex", "mcp", "add", SERVER, "--", *command]
144
+ raise ValueError(f"Client without MCP registration: {client!r}")
145
+
146
+
147
+ def remove_command(client):
148
+ """Official CLI command that removes the server registered by register_command."""
149
+ if client == "claude":
150
+ return ["claude", "mcp", "remove", SERVER, "--scope", "user"]
151
+ if client == "codex":
152
+ return ["codex", "mcp", "remove", SERVER]
153
+ raise ValueError(f"Client without MCP registration: {client!r}")
154
+
155
+
156
+ def status():
157
+ """Last outcome per client: {"claude": {"state": "registered"|"unavailable"|"kept", "reason", "at"}, ...}."""
158
+ try:
159
+ with open(STATUS_PATH, encoding="utf-8") as stream:
160
+ data = json.load(stream)
161
+ except FileNotFoundError:
162
+ return {}
163
+ if not isinstance(data, dict):
164
+ raise ValueError(f"{STATUS_PATH} is not a browser control status file: fix or delete it.")
165
+ return data
166
+
167
+
168
+ def _record(client, state, reason="", log=None):
169
+ if log and reason:
170
+ log(f"browser control {state} for {client}: {reason}")
171
+ data = status()
172
+ data[client] = {"state": state, "reason": reason, "at": time.time()}
173
+ os.makedirs(os.path.dirname(STATUS_PATH), exist_ok=True)
174
+ atomic_io.write_secret_text(STATUS_PATH, json.dumps(data, indent=1, ensure_ascii=False))
175
+ return data[client]
176
+
177
+
178
+ def ensure(client, log=None):
179
+ """Registers Playwright MCP in the client, or refreshes the registration Smart Tool made before (Node, Chromium
180
+ and install paths change between versions). A `playwright` server the user registered is kept. When the browser
181
+ cannot be driven on this machine (probe failed, CLI refused) records `unavailable` with the reason, removes a
182
+ registration of ours that would no longer work and returns: agents keep the read-only browsers of web_fetch and
183
+ web_search. Only an unreadable status file raises (ValueError). log (optional) also gets the reason."""
184
+ ours = status().get(client, {}).get("state") == "registered"
185
+ if registered().get(client) and not ours:
186
+ return _record(client, "kept", "a playwright MCP server you registered is kept")
187
+ try:
188
+ command = server_command()
189
+ problem = probe(command)
190
+ except (OSError, RuntimeError, subprocess.SubprocessError) as exc:
191
+ problem = str(exc)
192
+ try:
193
+ if ours and registered().get(client):
194
+ endpoint_sync.run_cli(remove_command(client))
195
+ if problem:
196
+ return _record(client, "unavailable", problem[:300], log)
197
+ endpoint_sync.run_cli(register_command(client, command))
198
+ except RuntimeError as exc:
199
+ return _record(client, "unavailable", str(exc)[:300], log)
200
+ if not registered().get(client):
201
+ return _record(client, "unavailable", "the client CLI finished, but the server did not appear in its configuration", log)
202
+ return _record(client, "registered")
package/endpoint_sync.py CHANGED
@@ -117,6 +117,15 @@ def register_command(client, base):
117
117
  def register(client, base):
118
118
  """Runs the client's own CLI (it owns its config file; ~/.claude.json is rewritten by Claude Code all the time)."""
119
119
  command = register_command(client, base)
120
+ run_cli(command)
121
+ if registrations().get(client) != f"{base}/mcp":
122
+ raise RuntimeError(f"'{' '.join(command)}' finished, but the registration did not appear in the client's configuration.")
123
+ return " ".join(command)
124
+
125
+
126
+ def run_cli(command):
127
+ """Runs a client CLI command (claude, codex) found on PATH, killing its whole shim tree after REGISTER_TIMEOUT_S.
128
+ Raises RuntimeError with the command and the CLI's own message when it is missing, hangs or exits non-zero."""
120
129
  executable = shutil.which(command[0])
121
130
  if not executable:
122
131
  raise RuntimeError(f"CLI '{command[0]}' not found in Smart Tool's PATH. Register it manually: {' '.join(command)}")
@@ -140,9 +149,6 @@ def register(client, base):
140
149
  if proc.returncode != 0:
141
150
  detail = (stderr or stdout).strip()[-400:]
142
151
  raise RuntimeError(f"'{' '.join(command)}' failed (code {proc.returncode}): {detail}")
143
- if registrations().get(client) != f"{base}/mcp":
144
- raise RuntimeError(f"'{' '.join(command)}' finished, but the registration did not appear in the client's configuration.")
145
- return " ".join(command)
146
152
 
147
153
 
148
154
  def known_port():
package/install.py CHANGED
@@ -291,6 +291,25 @@ def verify(target, python):
291
291
  raise RuntimeError(f"The active daemon (pid {pid}) does not belong to this installation: {command.strip()[:200]}")
292
292
 
293
293
 
294
+ def enable_browser_control(target, python):
295
+ """Registers Playwright MCP (browser_control.ensure) in each client where smart-tool is registered. Never fails the
296
+ installation: a machine that cannot drive a browser keeps the read-only browsers, and the outcome is printed."""
297
+ script = ("import json, browser_control, endpoint_sync; "
298
+ "print(json.dumps({c: browser_control.ensure(c) for c, url in endpoint_sync.registrations().items() if url}))")
299
+ try:
300
+ result = subprocess.run([str(python), "-c", script], cwd=target, capture_output=True, text=True,
301
+ encoding="utf-8", errors="replace", timeout=300)
302
+ outcome = json.loads(result.stdout.strip().splitlines()[-1]) if not result.returncode else None
303
+ except (OSError, subprocess.TimeoutExpired, ValueError, IndexError) as exc:
304
+ result, outcome = None, f"{type(exc).__name__}: {exc}"
305
+ if not isinstance(outcome, dict):
306
+ detail = outcome or ((result.stderr or result.stdout).strip().splitlines() or ["no output"])[-1]
307
+ print(f"Browser control not set up: {detail}")
308
+ return
309
+ for client, state in outcome.items():
310
+ print(f"Browser control in {client}: {state['state']}" + (f" ({state['reason']})" if state.get("reason") else ""))
311
+
312
+
294
313
  MIN_PYTHON = (3, 11)
295
314
 
296
315
 
@@ -363,6 +382,8 @@ def main():
363
382
  shutil.rmtree(backup, ignore_errors=True)
364
383
  raise SystemExit(1)
365
384
  shutil.rmtree(backup)
385
+ if not args.no_start:
386
+ enable_browser_control(target, python)
366
387
  print("Smart Tool ready." if not args.no_start else "Files installed; daemon not restarted (--no-start).")
367
388
 
368
389
 
@@ -30,7 +30,17 @@ OPEN_WEBSEARCH_CLI = NODE_DIR / "node_modules" / "open-websearch" / "build" / "i
30
30
  TYPESCRIPT_PARSER = NODE_DIR / "node_modules" / "typescript" / "lib" / "typescript.js"
31
31
  JAVA_PARSER = NODE_DIR / 'node_modules' / 'java-parser' / 'src' / 'index.js'
32
32
  ANGULAR_PARSER = NODE_DIR / 'node_modules' / '@angular' / 'compiler' / 'fesm2022' / 'compiler.mjs'
33
+ # Browser control for agents (browser_control.py): Microsoft's Playwright MCP over the Chromium installed below.
34
+ PLAYWRIGHT_MCP_CLI = NODE_DIR / "node_modules" / "@playwright" / "mcp" / "cli.js"
33
35
  NODE_RUNTIME = NODE_DIR / ".node-runtime"
36
+ # Moli renders the JavaScript pages web_fetch cannot read over HTTP; Chromium (Crawl4AI) stays as its fallback.
37
+ # Measured on 2026-10-10 over 21 such pages: 1.6 s and 90 MB peak (median) against 4.2 s and 581 MB for Chromium.
38
+ MOLI_VERSION = "1.1.15"
39
+ MOLI_FOLDER = f"moli-v{MOLI_VERSION}-x86_64-pc-windows-msvc"
40
+ MOLI_URL = f"https://github.com/lexmount/moli/releases/download/v{MOLI_VERSION}/moli-x86_64-pc-windows-msvc.zip"
41
+ MOLI_SHA256 = "64c00ce02d6db8e1c22c55e88fbcaaf3e7517e930f24c25ff46a457b421040c8"
42
+ MOLI_DIR = BROWSER_DIR / "moli"
43
+ MOLI_EXE = MOLI_DIR / "moli.exe"
34
44
 
35
45
 
36
46
  def _node_supported(node):
@@ -58,6 +68,49 @@ def _node_commands():
58
68
  return (node, npm) if node and npm and _node_supported(node) else (None, None)
59
69
 
60
70
 
71
+ def _download_verified(url, expected_sha256, path):
72
+ """Downloads url to path and fails when its SHA-256 differs from the published one."""
73
+ digest = hashlib.sha256()
74
+ with urllib.request.urlopen(url, timeout=60) as response, path.open("wb") as out:
75
+ while block := response.read(1024 * 1024):
76
+ digest.update(block)
77
+ out.write(block)
78
+ if digest.hexdigest().lower() != expected_sha256.lower():
79
+ raise RuntimeError(f"SHA-256 of {url} does not match the published checksum.")
80
+
81
+
82
+ def _extract_folder(zip_path, top, destination):
83
+ """Extracts the files under the archive's single top folder into destination, refusing any other path."""
84
+ destination.mkdir()
85
+ with zipfile.ZipFile(zip_path) as zf:
86
+ for member in zf.infolist():
87
+ parts = Path(member.filename.replace("\\", "/")).parts
88
+ if parts == (top,) and member.is_dir():
89
+ continue
90
+ if len(parts) < 2 or parts[0] != top or ".." in parts:
91
+ raise RuntimeError(f"{zip_path.name} contains an unexpected path: {member.filename}")
92
+ target = destination.joinpath(*parts[1:])
93
+ if member.is_dir():
94
+ target.mkdir(parents=True, exist_ok=True)
95
+ else:
96
+ target.parent.mkdir(parents=True, exist_ok=True)
97
+ with zf.open(member) as source, target.open("wb") as out:
98
+ shutil.copyfileobj(source, out)
99
+
100
+
101
+ def _replace_folder(unpacked, final, temp_dir):
102
+ """Swaps final for unpacked, putting the previous folder back if the swap fails."""
103
+ previous = temp_dir / "previous"
104
+ if final.exists():
105
+ os.replace(final, previous)
106
+ try:
107
+ os.replace(unpacked, final)
108
+ except OSError:
109
+ if previous.exists() and not final.exists():
110
+ os.replace(previous, final)
111
+ raise
112
+
113
+
61
114
  def _install_node():
62
115
  """Baixa o Node LTS oficial e confere SHA-256 antes de extrair."""
63
116
  print("Downloading Node LTS for the Smart Tool web runtime...", flush=True)
@@ -79,43 +132,38 @@ def _install_node():
79
132
  with tempfile.TemporaryDirectory(prefix="node-setup-", dir=NODE_DIR) as temp:
80
133
  temp_dir = Path(temp)
81
134
  zip_path = temp_dir / archive
82
- digest = hashlib.sha256()
83
- with urllib.request.urlopen(base + archive, timeout=60) as response, zip_path.open("wb") as out:
84
- while block := response.read(1024 * 1024):
85
- digest.update(block)
86
- out.write(block)
87
- if digest.hexdigest().lower() != expected.lower():
88
- raise RuntimeError("Node SHA-256 checksum does not match the official index.")
135
+ _download_verified(base + archive, expected, zip_path)
89
136
  unpacked = temp_dir / "runtime"
90
- unpacked.mkdir()
91
- with zipfile.ZipFile(zip_path) as zf:
92
- for member in zf.infolist():
93
- parts = Path(member.filename.replace("\\", "/")).parts
94
- if parts == (f"node-{version}-win-x64",) and member.is_dir():
95
- continue
96
- if len(parts) < 2 or parts[0] != f"node-{version}-win-x64" or ".." in parts:
97
- raise RuntimeError("Node archive contains an unexpected path.")
98
- destination = unpacked.joinpath(*parts[1:])
99
- if member.is_dir():
100
- destination.mkdir(parents=True, exist_ok=True)
101
- else:
102
- destination.parent.mkdir(parents=True, exist_ok=True)
103
- with zf.open(member) as source, destination.open("wb") as target:
104
- shutil.copyfileobj(source, target)
137
+ _extract_folder(zip_path, f"node-{version}-win-x64", unpacked)
105
138
  if not (unpacked / "node.exe").is_file() or not (unpacked / "npm.cmd").is_file():
106
139
  raise RuntimeError("Node package does not contain the expected executables.")
107
- previous = temp_dir / "previous-node-runtime"
108
- if NODE_RUNTIME.exists():
109
- os.replace(NODE_RUNTIME, previous)
110
- try:
111
- os.replace(unpacked, NODE_RUNTIME)
112
- except OSError:
113
- if previous.exists() and not NODE_RUNTIME.exists():
114
- os.replace(previous, NODE_RUNTIME)
115
- raise
140
+ _replace_folder(unpacked, NODE_RUNTIME, temp_dir)
116
141
  print(f"Node {version} installed inside Smart Tool.")
117
142
 
118
143
 
144
+ def _moli_ready():
145
+ try:
146
+ return MOLI_EXE.is_file() and (MOLI_DIR / "VERSION").read_text(encoding="utf-8").strip() == MOLI_VERSION
147
+ except OSError:
148
+ return False
149
+
150
+
151
+ def _install_moli():
152
+ """Downloads the pinned Moli release from GitHub, checks its SHA-256 and unpacks it into web_adapters/browser."""
153
+ print(f"Downloading Moli {MOLI_VERSION} (structured-first headless browser, 43 MB)...", flush=True)
154
+ BROWSER_DIR.mkdir(parents=True, exist_ok=True)
155
+ with tempfile.TemporaryDirectory(prefix="moli-setup-", dir=BROWSER_DIR) as temp:
156
+ temp_dir = Path(temp)
157
+ zip_path = temp_dir / "moli.zip"
158
+ _download_verified(MOLI_URL, MOLI_SHA256, zip_path)
159
+ unpacked = temp_dir / "moli"
160
+ _extract_folder(zip_path, MOLI_FOLDER, unpacked)
161
+ if not (unpacked / "moli.exe").is_file():
162
+ raise RuntimeError("The Moli package does not contain moli.exe.")
163
+ _replace_folder(unpacked, MOLI_DIR, temp_dir)
164
+ print(f"Moli {MOLI_VERSION} installed inside Smart Tool.")
165
+
166
+
119
167
  def _run(args, *, cwd=None, label="Web dependency", idle_s=600, stream=False, attempts=1):
120
168
  """Runs a step with UTF-8 output and closed stdin (a prompt fails at once instead of hanging); it is stopped only
121
169
  after idle_s without any output, and stream shows the output live."""
@@ -193,10 +241,12 @@ def _check():
193
241
  "typescript_parser": bool(node and TYPESCRIPT_PARSER.is_file()),
194
242
  "java_parser":bool(node and JAVA_PARSER.is_file()),
195
243
  "angular_parser":bool(node and ANGULAR_PARSER.is_file()),
244
+ "playwright_mcp": bool(node and PLAYWRIGHT_MCP_CLI.is_file()),
196
245
  "browser_python": BROWSER_PYTHON.is_file(),
197
246
  "browser_packages": False,
198
247
  "camoufox_browser": False,
199
248
  "chromium_browser": False,
249
+ "moli_browser": _moli_ready(),
200
250
  }
201
251
  if status["browser_python"]:
202
252
  try:
@@ -262,7 +312,8 @@ def install(reuse_browser_runtime=None):
262
312
  if not node or not npm:
263
313
  _install_node()
264
314
  node, npm = _node_commands()
265
- if not all(path.is_file() for path in (OPEN_WEBSEARCH_CLI,TYPESCRIPT_PARSER,JAVA_PARSER,ANGULAR_PARSER)):
315
+ if not all(path.is_file() for path in (OPEN_WEBSEARCH_CLI, TYPESCRIPT_PARSER, JAVA_PARSER, ANGULAR_PARSER,
316
+ PLAYWRIGHT_MCP_CLI)):
266
317
  _run([npm, "ci", "--omit=dev", "--no-audit", "--no-fund"], cwd=NODE_DIR,
267
318
  label="Web adapters and code parser via npm", idle_s=300)
268
319
  if reuse_browser_runtime:
@@ -286,10 +337,12 @@ def install(reuse_browser_runtime=None):
286
337
  idle_s=180, stream=True, attempts=3)
287
338
  _run([str(BROWSER_PYTHON), "-m", "playwright", "install", "chromium"],
288
339
  label="Chromium browser", idle_s=180, stream=True, attempts=3)
340
+ if not _moli_ready():
341
+ _install_moli()
289
342
  status = _check()
290
343
  if not all(status.values()):
291
344
  raise RuntimeError(f"Incomplete web runtime: {status}")
292
- print("Smart Tool runtime installed: Node, open-websearch, TypeScript, Java, Angular, Camoufox and Crawl4AI.")
345
+ print("Smart Tool runtime installed: Node, open-websearch, TypeScript, Java, Angular, Playwright MCP, Camoufox, Crawl4AI and Moli.")
293
346
 
294
347
 
295
348
  def main():
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@allansantos-dev/smart-tool",
3
- "version": "0.9.8",
3
+ "version": "0.9.10",
4
4
  "description": "Local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharper tools than their built-in search. Windows.",
5
5
  "license": "Apache-2.0",
6
6
  "author": "Allan Santos",
package/setup_ui.py CHANGED
@@ -20,6 +20,7 @@ import secrets
20
20
 
21
21
  import autostart
22
22
  import block_patterns
23
+ import browser_control
23
24
  import client_hooks
24
25
  import config
25
26
  import daemon_launcher
@@ -168,12 +169,17 @@ def handle_local_models_save(body):
168
169
 
169
170
  def handle_integration_get():
170
171
  registered = endpoint_sync.registrations()
172
+ try:
173
+ browser, browser_error = browser_control.status(), ""
174
+ except ValueError as exc:
175
+ browser, browser_error = {}, str(exc)
171
176
  clients = []
172
177
  for key, info in AGENT_CLIENTS.items():
173
178
  spec, state = client_hooks.CLIENTS[key], client_hooks.status(info)
174
179
  clients.append({"client": key, "label": spec["label"], "mcp_url": registered.get(key), "hook": state["installed"],
175
180
  "hook_error": state["error"], "file": spec["files"][0], "docs": spec["docs"],
176
- "register_command": " ".join(endpoint_sync.register_command(key, setup_origin()))})
181
+ "register_command": " ".join(endpoint_sync.register_command(key, setup_origin())),
182
+ "browser_control": browser.get(key), "browser_control_error": browser_error})
177
183
  try:
178
184
  mode, mode_error = config.hook_mode(), ""
179
185
  except ValueError as exc:
@@ -206,9 +212,12 @@ def handle_integration_get():
206
212
  def handle_integration_save(body):
207
213
  """`hook_mode` sets redirect/advise/off for every client; `register_mcp` runs the client's CLI; hook in two steps:
208
214
  `preview` shows the exact change, `install` writes it (with backup)."""
209
- if isinstance(body, dict) and body.get("action") == "block_pattern":
215
+ if isinstance(body, dict) and body.get("action") == "block_patterns":
216
+ ids = body.get("ids")
217
+ if not isinstance(ids, list) or not all(isinstance(i, str) for i in ids):
218
+ raise SetupError("ids must be a list of pattern ids.")
210
219
  try:
211
- block_patterns.decide_proposal(str(body.get("id") or ""), body.get("decision"))
220
+ block_patterns.decide_proposals(ids, body.get("decision"))
212
221
  except (OSError, ValueError) as exc:
213
222
  raise SetupError(str(exc))
214
223
  return handle_integration_get()
@@ -228,14 +237,18 @@ def handle_integration_save(body):
228
237
  info = AGENT_CLIENTS[body["client"]]
229
238
  try:
230
239
  if body.get("action") == "register_mcp":
231
- return {"command": endpoint_sync.register(body["client"], setup_origin()), **handle_integration_get()}
240
+ command = endpoint_sync.register(body["client"], setup_origin())
241
+ return {"command": command, "browser_control": browser_control.ensure(body["client"]),
242
+ **handle_integration_get()}
243
+ if body.get("action") == "browser_control":
244
+ return {"browser_control": browser_control.ensure(body["client"]), **handle_integration_get()}
232
245
  if body.get("action") == "preview":
233
246
  return client_hooks.preview(info)
234
247
  if body.get("action") == "install":
235
248
  return {**client_hooks.install(info, body.get("token"), True), **handle_integration_get()}
236
249
  except (ValueError, RuntimeError) as exc:
237
250
  raise SetupError(str(exc))
238
- raise SetupError("Invalid action: use hook_mode, register_mcp, preview or install.")
251
+ raise SetupError("Invalid action: use hook_mode, register_mcp, browser_control, preview or install.")
239
252
 
240
253
 
241
254
  def handle_providers_get():
@@ -462,6 +475,29 @@ _PAGE_TEMPLATE = """<!doctype html>
462
475
  .provider-head { display: flex; justify-content: space-between; align-items: baseline; gap: 10px; margin-bottom: 8px; }
463
476
  .provider-head strong { font: 600 .98rem var(--display); letter-spacing: .02em; }
464
477
  .blocked { color: var(--muted); font-size: .88rem; border: 1px dashed var(--line); border-radius: 10px; padding: 14px 16px; }
478
+ .pattern-panel, .pattern-blocking { margin-top: 16px; border: 1px solid var(--line); border-radius: 10px; padding: 14px 16px; }
479
+ .pattern-head { display: flex; flex-direction: column; gap: 2px; }
480
+ .pattern-tools { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 10px; margin-top: 12px; }
481
+ .pattern-bulk { margin-top: 10px; }
482
+ .pattern-list { list-style: none; margin: 10px 0 0; padding: 0; border-top: 1px solid var(--line); }
483
+ .pattern-row { display: grid; grid-template-columns: auto minmax(0, 1fr); gap: 10px; align-items: start;
484
+ padding: 10px 0; border-bottom: 1px solid var(--line); }
485
+ .pattern-blocking .pattern-row { grid-template-columns: minmax(0, 1fr); }
486
+ .pattern-row input[type=checkbox] { margin-top: 6px; }
487
+ .pattern-body summary { display: flex; gap: 8px; align-items: baseline; cursor: pointer; min-width: 0; }
488
+ .pattern-tool { font: 600 .74rem var(--display); letter-spacing: .06em; text-transform: uppercase; color: var(--muted); flex: none; }
489
+ .pattern-code { font: .78rem/1.4 var(--mono); overflow-wrap: anywhere; min-width: 0; display: -webkit-box;
490
+ -webkit-line-clamp: 2; -webkit-box-orient: vertical; overflow: hidden; }
491
+ .pattern-body[open] .pattern-code { -webkit-line-clamp: unset; }
492
+ .pattern-main { min-width: 0; }
493
+ .pattern-meta { display: flex; justify-content: space-between; align-items: center; gap: 10px; margin-top: 6px; flex-wrap: wrap; }
494
+ .pattern-detail { font-size: .84rem; padding: 4px 0 2px; }
495
+ .pattern-detail p { margin: 4px 0; }
496
+ .pattern-reach { font: .78rem var(--mono); color: var(--muted); white-space: nowrap; }
497
+ .pattern-actions { display: flex; gap: 6px; }
498
+ .pattern-actions button { padding: 4px 10px; font-size: .78rem; }
499
+ .pattern-pager { justify-content: space-between; }
500
+ .pattern-blocking > summary { cursor: pointer; font: 600 .9rem var(--display); }
465
501
 
466
502
  body:not(.ready) .station[data-live="1"] .wire::after { transform: scaleX(0); }
467
503
  @media (max-width: 900px) { .panels { grid-template-columns: minmax(0, 1fr); } }
@@ -897,6 +933,14 @@ $("btn-save-local-models").addEventListener("click", async () => {
897
933
  }
898
934
  });
899
935
 
936
+ function browserText(item) {
937
+ if (item.browser_control_error) return "Browser control status unreadable: " + item.browser_control_error;
938
+ const state = item.browser_control || {};
939
+ if (state.state === "registered") return "Browser control on: Playwright MCP drives an isolated, headless Chromium";
940
+ if (state.state === "kept") return "Browser control: your own playwright MCP server is kept";
941
+ if (state.state === "unavailable") return "Browser control unavailable on this machine (" + state.reason + "); pages are still read by web_fetch";
942
+ return "Browser control is set up when the MCP is registered";
943
+ }
900
944
  async function loadAgents() {
901
945
  try {
902
946
  const resp = await fetch("/setup/integration");
@@ -918,13 +962,28 @@ async function loadAgents() {
918
962
  row.className = "provider";
919
963
  const mcp = item.mcp_url ? "MCP registered: " + item.mcp_url : "smart-tool MCP not registered in this client";
920
964
  const hook = item.hook_error ? "Hook unreadable: " + item.hook_error : item.hook ? "Hook installed and up to date" : "Hook missing or outdated";
921
- row.innerHTML = `<div class="provider-head"><strong></strong><span class="state"></span></div><p class="hint mcp"></p><p class="hint mono cmd"></p><p class="hint file"></p>
922
- <div class="actions"><button type="button" class="register">Register MCP</button><button type="button" class="hook">Install or update hook</button></div>
965
+ row.innerHTML = `<div class="provider-head"><strong></strong><span class="state"></span></div><p class="hint mcp"></p><p class="hint browser"></p><p class="hint mono cmd"></p><p class="hint file"></p>
966
+ <div class="actions"><button type="button" class="register">Register MCP</button><button type="button" class="hook">Install or update hook</button><button type="button" class="quiet browser-retry">Set up browser control</button></div>
923
967
  <div class="plan" hidden><span class="hint plan-file"></span><pre class="code-preview"></pre>
924
968
  <div class="actions"><button type="button" class="confirm">Confirm and write</button><button type="button" class="quiet cancel">Cancel</button></div></div>`;
925
969
  row.querySelector("strong").textContent = item.label;
926
970
  row.querySelector(".state").textContent = hook;
927
971
  row.querySelector(".mcp").textContent = mcp;
972
+ row.querySelector(".browser").textContent = browserText(item);
973
+ const retry = row.querySelector(".browser-retry");
974
+ retry.hidden = !item.mcp_url || ["registered", "kept"].includes((item.browser_control || {}).state);
975
+ retry.addEventListener("click", async () => {
976
+ retry.disabled = true;
977
+ setMsg("agents-msg", "", "Checking whether agents can drive a browser here…");
978
+ try {
979
+ const result = await post("/setup/integration", { client: item.client, action: "browser_control" });
980
+ setMsg("agents-msg", "ok", item.label + ": " + browserText({ browser_control: result.browser_control }));
981
+ await loadAgents();
982
+ } catch (error) {
983
+ setMsg("agents-msg", "error", error.message);
984
+ retry.disabled = false;
985
+ }
986
+ });
928
987
  row.querySelector(".file").textContent = item.file;
929
988
  row.querySelector(".cmd").textContent = item.mcp_url ? "" : "Or in a terminal: " + item.register_command;
930
989
  const register = row.querySelector(".register");
@@ -934,7 +993,7 @@ async function loadAgents() {
934
993
  setMsg("agents-msg", "", "Registering the MCP in " + item.label + "…");
935
994
  try {
936
995
  const result = await post("/setup/integration", { client: item.client, action: "register_mcp" });
937
- setMsg("agents-msg", "ok", "MCP registered with: " + result.command + ". Reconnect open sessions (/mcp).");
996
+ setMsg("agents-msg", "ok", "MCP registered with: " + result.command + ". " + browserText({ browser_control: result.browser_control }) + ". Reconnect open sessions (/mcp).");
938
997
  await loadAgents();
939
998
  } catch (error) {
940
999
  setMsg("agents-msg", "error", error.message);
@@ -973,49 +1032,163 @@ async function loadAgents() {
973
1032
  setMsg("agents-msg", "error", "Agent status unavailable: " + error.message);
974
1033
  }
975
1034
  }
1035
+ const PATTERN_PAGE_SIZE = 8;
1036
+ const patternView = { page: 0, filter: "", sort: "impact", selected: new Set(), open: false };
1037
+ let lastPatterns = null;
1038
+ function patternPage(items, view) {
1039
+ const needle = view.filter.trim().toLowerCase();
1040
+ const matched = items.filter((item) => !needle || [item.tool, item.pattern, item.reason, item.example]
1041
+ .some((text) => String(text || "").toLowerCase().includes(needle)));
1042
+ const sorted = matched.slice().sort(view.sort === "newest"
1043
+ ? (a, b) => (b.proposed_at || 0) - (a.proposed_at || 0)
1044
+ : (a, b) => (b.history_matches || 0) - (a.history_matches || 0) || (b.proposed_at || 0) - (a.proposed_at || 0));
1045
+ const pages = Math.max(1, Math.ceil(sorted.length / PATTERN_PAGE_SIZE));
1046
+ const page = Math.min(Math.max(0, view.page), pages - 1);
1047
+ return { rows: sorted.slice(page * PATTERN_PAGE_SIZE, (page + 1) * PATTERN_PAGE_SIZE), page, pages,
1048
+ matched: sorted.length, total: items.length,
1049
+ calls: items.reduce((sum, item) => sum + (item.history_matches || 0), 0) };
1050
+ }
1051
+ function patternEl(tag, className, text) {
1052
+ const el = document.createElement(tag);
1053
+ if (className) el.className = className;
1054
+ if (text !== undefined) el.textContent = text;
1055
+ return el;
1056
+ }
1057
+ async function decidePatterns(ids, decision, label) {
1058
+ try {
1059
+ const result = await post("/setup/integration", { action: "block_patterns", ids, decision });
1060
+ for (const id of ids) patternView.selected.delete(id);
1061
+ renderPatterns(result.block_patterns);
1062
+ setMsg("agents-msg", "ok", label + ": " + ids.length + " pattern(s).");
1063
+ } catch (error) {
1064
+ setMsg("agents-msg", "error", error.message);
1065
+ }
1066
+ }
1067
+ function patternRow(item, actions, selectable) {
1068
+ const row = patternEl("li", "pattern-row");
1069
+ if (selectable) {
1070
+ const check = patternEl("input");
1071
+ check.type = "checkbox";
1072
+ check.checked = patternView.selected.has(item.id);
1073
+ check.setAttribute("aria-label", "Select " + item.pattern);
1074
+ check.addEventListener("change", () => {
1075
+ if (check.checked) patternView.selected.add(item.id); else patternView.selected.delete(item.id);
1076
+ renderPatterns(lastPatterns);
1077
+ });
1078
+ row.appendChild(check);
1079
+ }
1080
+ const body = patternEl("details", "pattern-body");
1081
+ const summary = patternEl("summary");
1082
+ const code = patternEl("code", "pattern-code", item.pattern);
1083
+ code.title = item.pattern;
1084
+ summary.append(patternEl("span", "pattern-tool", item.tool), code);
1085
+ const detail = patternEl("div", "pattern-detail");
1086
+ detail.append(patternEl("p", "", item.reason), patternEl("p", "hint mono", "Example: " + item.example),
1087
+ patternEl("p", "hint mono", item.pattern));
1088
+ body.append(summary, detail);
1089
+ const reach = patternEl("span", "pattern-reach", item.history_total ? item.history_matches + " / " + item.history_total : "–");
1090
+ reach.title = item.history_total ? "Would have blocked " + item.history_matches + " of " + item.history_total + " logged " + item.tool + " calls" : "No logged calls to compare";
1091
+ const buttons = patternEl("span", "pattern-actions");
1092
+ for (const [decision, label] of actions) {
1093
+ const button = patternEl("button", "quiet", label);
1094
+ button.type = "button";
1095
+ button.addEventListener("click", () => decidePatterns([item.id], decision, label));
1096
+ buttons.appendChild(button);
1097
+ }
1098
+ const meta = patternEl("div", "pattern-meta");
1099
+ meta.append(reach, buttons);
1100
+ const main = patternEl("div", "pattern-main");
1101
+ main.append(body, meta);
1102
+ row.appendChild(main);
1103
+ return row;
1104
+ }
976
1105
  function renderPatterns(patterns) {
1106
+ lastPatterns = patterns;
977
1107
  const box = $("patterns-list");
978
1108
  box.textContent = "";
979
- if (patterns.error) { const p = document.createElement("p"); p.className = "msg error"; p.textContent = patterns.error; box.appendChild(p); return; }
980
- const groups = [["proposals", "Proposed", [["accept", "Accept"], ["reject", "Reject"]]], ["accepted", "Blocking", [["remove", "Remove"]]]];
981
- for (const [key, title, actions] of groups) {
982
- if (!patterns[key].length) continue;
983
- const head = document.createElement("h3");
984
- head.textContent = title + " (" + patterns[key].length + ")";
985
- box.appendChild(head);
986
- for (const item of patterns[key]) {
987
- const row = document.createElement("div");
988
- row.className = "provider";
989
- const code = document.createElement("code");
990
- code.textContent = item.tool + ": " + item.pattern;
991
- const why = document.createElement("p");
992
- const seen = item.history_total ? " Would have blocked " + item.history_matches + " of " + item.history_total + " logged " + item.tool + " calls." : "";
993
- why.textContent = item.reason + seen;
994
- const example = document.createElement("p");
995
- example.className = "hint";
996
- example.textContent = "Example: " + item.example;
997
- row.append(code, why, example);
998
- const buttons = document.createElement("div");
999
- buttons.className = "actions";
1000
- for (const [decision, label] of actions) {
1001
- const button = document.createElement("button");
1002
- button.type = "button";
1003
- button.textContent = label;
1004
- button.addEventListener("click", async () => {
1005
- try {
1006
- renderPatterns((await post("/setup/integration", { action: "block_pattern", id: item.id, decision })).block_patterns);
1007
- setMsg("agents-msg", "ok", label + ": " + item.pattern);
1008
- } catch (error) {
1009
- setMsg("agents-msg", "error", error.message);
1010
- }
1011
- });
1012
- buttons.appendChild(button);
1013
- }
1014
- row.appendChild(buttons);
1015
- box.appendChild(row);
1109
+ if (patterns.error) { box.appendChild(patternEl("p", "msg error", patterns.error)); return; }
1110
+ const proposals = patterns.proposals;
1111
+ const live = new Set(proposals.map((item) => item.id));
1112
+ for (const id of [...patternView.selected]) if (!live.has(id)) patternView.selected.delete(id);
1113
+ if (proposals.length) {
1114
+ const view = patternPage(proposals, patternView);
1115
+ patternView.page = view.page;
1116
+ const panel = patternEl("div", "pattern-panel");
1117
+ const head = patternEl("div", "pattern-head");
1118
+ head.append(patternEl("strong", "", proposals.length + " proposed pattern(s) waiting"),
1119
+ patternEl("span", "hint", "Together they would have blocked " + view.calls + " logged call(s). Open a row for its reason and example."));
1120
+ const tools = patternEl("div", "pattern-tools");
1121
+ const filter = patternEl("input");
1122
+ filter.type = "text";
1123
+ filter.placeholder = "Filter by pattern, reason or example";
1124
+ filter.value = patternView.filter;
1125
+ filter.setAttribute("aria-label", "Filter proposed patterns");
1126
+ filter.addEventListener("input", () => {
1127
+ patternView.filter = filter.value;
1128
+ patternView.page = 0;
1129
+ renderPatterns(lastPatterns);
1130
+ const again = document.querySelector(".pattern-tools input[type=text]");
1131
+ if (again) { again.focus(); again.setSelectionRange(again.value.length, again.value.length); }
1132
+ });
1133
+ const sort = patternEl("select");
1134
+ sort.setAttribute("aria-label", "Order of proposed patterns");
1135
+ for (const [value, label] of [["impact", "Most blocked calls first"], ["newest", "Newest first"]]) {
1136
+ const option = patternEl("option", "", label);
1137
+ option.value = value;
1138
+ sort.appendChild(option);
1016
1139
  }
1140
+ sort.value = patternView.sort;
1141
+ sort.addEventListener("change", () => { patternView.sort = sort.value; patternView.page = 0; renderPatterns(lastPatterns); });
1142
+ tools.append(filter, sort);
1143
+ const bulk = patternEl("div", "actions pattern-bulk");
1144
+ const pageIds = view.rows.map((item) => item.id);
1145
+ const allOnPage = pageIds.length > 0 && pageIds.every((id) => patternView.selected.has(id));
1146
+ const selectPage = patternEl("button", "quiet", allOnPage ? "Clear this page" : "Select this page");
1147
+ selectPage.type = "button";
1148
+ selectPage.disabled = !pageIds.length;
1149
+ selectPage.addEventListener("click", () => {
1150
+ for (const id of pageIds) if (allOnPage) patternView.selected.delete(id); else patternView.selected.add(id);
1151
+ renderPatterns(lastPatterns);
1152
+ });
1153
+ const count = patternView.selected.size;
1154
+ const accept = patternEl("button", "", "Accept selected (" + count + ")");
1155
+ const reject = patternEl("button", "quiet", "Reject selected (" + count + ")");
1156
+ for (const [button, decision, label] of [[accept, "accept", "Accepted"], [reject, "reject", "Rejected"]]) {
1157
+ button.type = "button";
1158
+ button.disabled = !count;
1159
+ button.addEventListener("click", () => decidePatterns([...patternView.selected], decision, label));
1160
+ }
1161
+ bulk.append(selectPage, accept, reject);
1162
+ const list = patternEl("ul", "pattern-list");
1163
+ for (const item of view.rows) list.appendChild(patternRow(item, [["accept", "Accept"], ["reject", "Reject"]], true));
1164
+ if (!view.rows.length) list.appendChild(patternEl("li", "hint", "No proposal matches the filter."));
1165
+ const pager = patternEl("div", "actions pattern-pager");
1166
+ const prev = patternEl("button", "quiet", "Previous");
1167
+ const next = patternEl("button", "quiet", "Next");
1168
+ prev.type = "button";
1169
+ next.type = "button";
1170
+ prev.disabled = view.page === 0;
1171
+ next.disabled = view.page >= view.pages - 1;
1172
+ prev.addEventListener("click", () => { patternView.page -= 1; renderPatterns(lastPatterns); });
1173
+ next.addEventListener("click", () => { patternView.page += 1; renderPatterns(lastPatterns); });
1174
+ const first = view.page * PATTERN_PAGE_SIZE;
1175
+ const shown = view.matched ? (first + 1) + "–" + (first + view.rows.length) : "0";
1176
+ const filtered = view.matched < view.total ? " (filtered from " + view.total + ")" : "";
1177
+ pager.append(prev, patternEl("span", "hint", shown + " of " + view.matched + filtered + " · page " + (view.page + 1) + " of " + view.pages), next);
1178
+ panel.append(head, tools, bulk, list, pager);
1179
+ box.appendChild(panel);
1180
+ }
1181
+ if (patterns.accepted.length) {
1182
+ const blocking = patternEl("details", "pattern-blocking");
1183
+ blocking.open = patternView.open;
1184
+ blocking.addEventListener("toggle", () => { patternView.open = blocking.open; });
1185
+ blocking.appendChild(patternEl("summary", "", "Blocking patterns (" + patterns.accepted.length + ")"));
1186
+ const list = patternEl("ul", "pattern-list");
1187
+ for (const item of patterns.accepted) list.appendChild(patternRow(item, [["remove", "Remove"]], false));
1188
+ blocking.appendChild(list);
1189
+ box.appendChild(blocking);
1017
1190
  }
1018
- if (patterns.rejected) { const p = document.createElement("p"); p.className = "hint"; p.textContent = patterns.rejected + " rejected proposal(s) are not offered again."; box.appendChild(p); }
1191
+ if (patterns.rejected) box.appendChild(patternEl("p", "hint", patterns.rejected + " rejected proposal(s) are not offered again."));
1019
1192
  }
1020
1193
  loadAgents();
1021
1194
  $("btn-save-pattern-proposals").addEventListener("click", async () => {
@@ -37,6 +37,7 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
37
37
  sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
38
38
  import capabilities
39
39
  import client_hooks
40
+ import install_runtime
40
41
  import integration_metrics
41
42
  import config
42
43
  import freshness
@@ -3187,17 +3188,52 @@ def _tool_result(text, is_error=False):
3187
3188
 
3188
3189
  WEB_FETCH_TIMEOUT_S = 90
3189
3190
  WEB_FETCH_RENDER_TIMEOUT_S = 60
3191
+ MOLI_RENDER_TIMEOUT_S = 25
3192
+ # Engine of the last render on this request's thread: web_fetch.read_page calls _render_page on the handler's thread.
3193
+ _RENDER = threading.local()
3190
3194
  WEB_FETCH_VECTOR_ROOT = os.path.join(paths.DATA_DIR, "web-fetch")
3191
3195
 
3192
3196
 
3197
+ def _moli_markdown(url, timeout):
3198
+ """Markdown of a JavaScript page rendered by Moli (layout and paint only when the page needs them)."""
3199
+ if not install_runtime.MOLI_EXE.is_file():
3200
+ raise RuntimeError(f"Moli is not installed ({install_runtime.MOLI_EXE}); reinstall Smart Tool.")
3201
+ done = subprocess.run([str(install_runtime.MOLI_EXE), "fetch", "--dump", "markdown", url], capture_output=True,
3202
+ timeout=timeout, creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0))
3203
+ if done.returncode:
3204
+ raise RuntimeError(f"moli exited {done.returncode}: {done.stderr.decode('utf-8', 'replace').strip()[-300:]}")
3205
+ return done.stdout.decode("utf-8", "replace")
3206
+
3207
+
3193
3208
  def _render_page(url, deadline):
3209
+ """Moli first; Chromium through Crawl4AI only when Moli fails or comes back thin. Measured on 2026-10-10 over 21
3210
+ pages HTTP could not read: Moli 1.6 s and 90 MB peak (median), Chromium 4.2 s and 581 MB; Moli read 14, Chromium
3211
+ 16, both together 17 (only Chromium read YouTube, Google Maps and Airbnb). The engine that answered goes to the
3212
+ web_fetch metric through _RENDER."""
3194
3213
  left = deadline - time.monotonic()
3195
3214
  if left < 5:
3196
3215
  return ""
3216
+ _RENDER.engine, _RENDER.moli_error = None, None
3217
+ try:
3218
+ markdown = _moli_markdown(url, min(MOLI_RENDER_TIMEOUT_S, left - 2))
3219
+ except (OSError, RuntimeError, subprocess.TimeoutExpired) as exc:
3220
+ markdown, _RENDER.moli_error = "", f"{type(exc).__name__}: {exc}"[:200]
3221
+ _log(f"web_fetch Moli render failed, trying Chromium: {_RENDER.moli_error}")
3222
+ if not web_fetch.thin(markdown):
3223
+ _RENDER.engine = "moli"
3224
+ return markdown
3225
+ left = deadline - time.monotonic()
3226
+ if left < 5:
3227
+ return markdown
3197
3228
  raw = _run_search_script(CAMOUFOX_VENV_PYTHON, CRAWL4AI_SEARCH_SCRIPT, [url, "--max-pages", "1"],
3198
3229
  min(WEB_FETCH_RENDER_TIMEOUT_S, left))
3199
3230
  pages = raw.get("pages") if isinstance(raw, dict) else None
3200
- return (pages[0].get("markdown") or "") if pages else ""
3231
+ chromium = (pages[0].get("markdown") or "") if pages else ""
3232
+ if len(chromium.strip()) > len(markdown.strip()):
3233
+ _RENDER.engine = "chromium"
3234
+ return chromium
3235
+ _RENDER.engine = "moli"
3236
+ return markdown
3201
3237
 
3202
3238
 
3203
3239
  def _web_fetch_embed(texts, deadline):
@@ -3243,6 +3279,7 @@ def _handle_web_fetch(arguments):
3243
3279
  if problem:
3244
3280
  raise RuntimeError(f"Gateway model unavailable ({problem}).")
3245
3281
  deadline = started + WEB_FETCH_TIMEOUT_S
3282
+ _RENDER.engine, _RENDER.moli_error = None, None
3246
3283
  page = web_fetch.read_page(url, deadline, _render_page)
3247
3284
  selected, pieces = web_fetch.select(page["text"], prompt, lambda texts: _web_fetch_embed(texts, deadline))
3248
3285
  text = web_fetch.answer(config.load_config()["router_model"], prompt, {**page, "text": selected}, deadline)
@@ -3252,6 +3289,7 @@ def _handle_web_fetch(arguments):
3252
3289
  raise RuntimeError(f"web_fetch failed: {str(exc).rstrip('.')}. The native WebFetch is allowed for this URL for "
3253
3290
  f"{web_fetch.FAILED_TTL_S // 60} min.") from None
3254
3291
  _metric("web_fetch", cached=page["cached"], rendered=page["rendered"], render_error=page.get("render_error"),
3292
+ render_engine=_RENDER.engine if page["rendered"] else None, moli_error=_RENDER.moli_error,
3255
3293
  page_chars=page["chars"], read_chars=len(selected), pieces=pieces,
3256
3294
  answer_chars=len(text), elapsed_s=round(time.monotonic() - started, 2))
3257
3295
  origin = "from cache" if page["cached"] else "rendered in the browser" if page["rendered"] else "downloaded now"
package/version.py CHANGED
@@ -1,7 +1,7 @@
1
1
  """Single source of the Smart Tool version and of the User-Agent sent to public APIs."""
2
2
  import re
3
3
 
4
- VERSION = "0.9.8"
4
+ VERSION = "0.9.10"
5
5
  _CONTACT_RE = re.compile(r"^(?:[^\s@()<>;]+@[^\s@()<>;]+\.[^\s@()<>;]+|https?://[^\s()<>;]+)$")
6
6
 
7
7
 
@@ -1,14 +1,15 @@
1
1
  {
2
2
  "name": "smart-tool-web-adapters",
3
- "version": "1.0.0",
3
+ "version": "0.9.0",
4
4
  "lockfileVersion": 3,
5
5
  "requires": true,
6
6
  "packages": {
7
7
  "": {
8
8
  "name": "smart-tool-web-adapters",
9
- "version": "1.0.0",
9
+ "version": "0.9.0",
10
10
  "dependencies": {
11
11
  "@angular/compiler": "20.3.0",
12
+ "@playwright/mcp": "0.0.83",
12
13
  "java-parser": "3.0.1",
13
14
  "open-websearch": "2.1.11",
14
15
  "typescript": "5.9.3"
@@ -588,6 +589,22 @@
588
589
  "node": ">=14.0.0"
589
590
  }
590
591
  },
592
+ "node_modules/@playwright/mcp": {
593
+ "version": "0.0.83",
594
+ "resolved": "https://registry.npmjs.org/@playwright/mcp/-/mcp-0.0.83.tgz",
595
+ "integrity": "sha512-oNcl+Ae2/IAjhfPeP46BfIkSakfmprY+aOtkv5MjrQ4lPav4/yNtPhL0iq8SlIM90oApWgBDUxaNKvktazUKOg==",
596
+ "license": "Apache-2.0",
597
+ "dependencies": {
598
+ "playwright": "1.64.0-alpha-1790635538000",
599
+ "playwright-core": "1.64.0-alpha-1790635538000"
600
+ },
601
+ "bin": {
602
+ "playwright-mcp": "cli.js"
603
+ },
604
+ "engines": {
605
+ "node": ">=18"
606
+ }
607
+ },
591
608
  "node_modules/@types/cheerio": {
592
609
  "version": "0.22.35",
593
610
  "resolved": "https://registry.npmjs.org/@types/cheerio/-/cheerio-0.22.35.tgz",
@@ -2087,6 +2104,33 @@
2087
2104
  "node": ">=16.20.0"
2088
2105
  }
2089
2106
  },
2107
+ "node_modules/playwright": {
2108
+ "version": "1.64.0-alpha-1790635538000",
2109
+ "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.64.0-alpha-1790635538000.tgz",
2110
+ "integrity": "sha512-/5XDUMxpOd/9AojJtWDFIRV3EJZNQ0AwvK69wgt3CPzy8wTgiBG9pX45VfxtqtvIRGwfWzMhrvawNKreG69Snw==",
2111
+ "license": "Apache-2.0",
2112
+ "dependencies": {
2113
+ "playwright-core": "1.64.0-alpha-1790635538000"
2114
+ },
2115
+ "bin": {
2116
+ "playwright": "cli.js"
2117
+ },
2118
+ "engines": {
2119
+ "node": ">=20"
2120
+ }
2121
+ },
2122
+ "node_modules/playwright-core": {
2123
+ "version": "1.64.0-alpha-1790635538000",
2124
+ "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.64.0-alpha-1790635538000.tgz",
2125
+ "integrity": "sha512-pNwaXirhXMRLaRQs4NQ18EpTdtDoFwyPH9FOaaDC7YXG4hpwE2xmwCic/1srbuExVQC7zT9LJSJpDBAru+Vo9A==",
2126
+ "license": "Apache-2.0",
2127
+ "bin": {
2128
+ "playwright-core": "cli.js"
2129
+ },
2130
+ "engines": {
2131
+ "node": ">=20"
2132
+ }
2133
+ },
2090
2134
  "node_modules/proxy-addr": {
2091
2135
  "version": "2.0.8",
2092
2136
  "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.8.tgz",
@@ -5,6 +5,7 @@
5
5
  "description": "Adaptadores Node empacotados com a Smart Tool",
6
6
  "dependencies": {
7
7
  "@angular/compiler": "20.3.0",
8
+ "@playwright/mcp": "0.0.83",
8
9
  "java-parser": "3.0.1",
9
10
  "open-websearch": "2.1.11",
10
11
  "typescript": "5.9.3"