@allansantos-dev/smart-tool 0.9.78 → 0.9.80

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,84 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.9.80 - beta
4
+
5
+ - The document converter keeps its results in one folder per version, `%LOCALAPPDATA%\SmartTool\cache\<version>\convert`,
6
+ named by a hash of the file content, the engine and the OCR flag: the same file read again is answered from that folder
7
+ without starting the engine, a second run creates no file, two runs at once make one, a failed read leaves nothing, and
8
+ the folders of older versions are deleted the first time it is used. It no longer makes a new temporary folder in the
9
+ system temp for every read. Tests: `tests/test_document_converter.py`.
10
+ - The edit hook also notes a credential an edit writes, in any file: only on the lines the edit adds or changes, and only
11
+ for shapes that cannot be anything else (GitHub, Slack, Stripe live, Anthropic and npm tokens, Google API keys, AWS
12
+ access key ids and secret key lines, a private-key header at the start of a line). Lines that say they are an example are
13
+ skipped, the note names the file and line and never repeats the value, and it never blocks. Tests:
14
+ `tests/test_credential_check.py`.
15
+ - `project_manage action=pr_impact` adds two sections: the files that usually change in the same commits as the diff's files
16
+ and are missing from it (the history coupling of `doc_insights`, import-linked pairs left out), and the decision notes and
17
+ commit messages that name a touched function. A single-word function name counts only as a code reference (backticks or a
18
+ call), because words such as `load` or `report` are ordinary prose: measured on this repository, the bare-word match
19
+ returned unrelated notes for most single-word names. Tests: `tests/test_pr_impact.py`, `tests/test_code_health.py`.
20
+ - The dependency check reads `Cargo.toml` (crates.io: the dependency, dev-dependency and build-dependency tables, `target.*`
21
+ and `workspace.dependencies`; path, git and `workspace = true` entries are not registry packages) and `pom.xml` (Maven:
22
+ `groupId:artifactId`, `${property}` versions resolved from the pom's own properties; a range or a version inherited from a
23
+ parent or BOM is reported as not verified), on an edit and in the audit. The OSV names `crates.io` and `Maven` come from
24
+ the OSV schema. Tests: `tests/test_dependency_manifests.py`.
25
+ - Web search page: a table of who asked the recent searches, one row per client and project folder (sessions, calls,
26
+ failures, last call, how the requester was attributed), read from the log the search tiers already write; the text of a
27
+ search is never recorded. Tests: `tests/test_web_search_requesters.py`.
28
+ - Startup page: an automatic update card with the switch (the same `auto_update` key as the tray menu), the installed
29
+ version, the last npm check, the last update, a failed automatic try and the manual command, read from the state file with
30
+ no network call; the switch is refused while `SMART_TOOL_AUTO_UPDATE=off` is set, as in the tray. Tests:
31
+ `tests/test_update_card.py`.
32
+ - The browsers Smart Tool starts with Chromium (the Playwright MCP server the agent drives, its start-up probe, the
33
+ Tier 4 crawl) no longer scan USB devices: they carry `WebUsbDeviceDetection` in their `--disable-features`. The scan
34
+ ran a few seconds after launch and froze a USB microphone that was recording. Chromium keeps only the last
35
+ `--disable-features` switch and Playwright appends the caller's arguments after its own list, so the switch repeats
36
+ the list of the Playwright bundle in use and adds the feature; a bundle where that list cannot be found fails loudly.
37
+ Existing installs pick it up on the next update. Tests: `tests/test_chromium_flags.py`
38
+ (`docs/research/2026-11-17-webusb-do-navegador-de-automacao.md`).
39
+ - `web_fetch` reads Word, PowerPoint and Excel files by URL (docx, pptx, xlsx): by content type, or by the extension
40
+ in the URL when the server sends a generic binary type, through the extractor the index uses and its archive guards.
41
+ A file that cannot be read says why. Tests: `tests/test_web_fetch.py`.
42
+ - `project_manage action=deps` lists the OSV advisories of every dependency the project's manifests declare, in
43
+ batches of 100 per ecosystem (the edit notice shares the cache). `smart_search mode=dossier history=true` also
44
+ searches commit messages, and `smart_search decision_notes=true` appends the project's decision notes to a search
45
+ answer; both default to off. Tests: `tests/test_sister_candidates.py`.
46
+ - `read_document` also reads HTML (`.html`, `.htm`), CSV and EPUB files with no new dependency: the standard HTML and
47
+ CSV parsers, and the archive guards of the Office formats for the epub package (its chapters are read in spine
48
+ order). A page or book in an encoding the reader cannot use, or over the text budget, fails with the reason. Tests:
49
+ `tests/test_document_formats.py`.
50
+ - The dependency check reads `pyproject.toml` (`project.dependencies` and `project.optional-dependencies`; Poetry
51
+ tables are not read), both on an edit and in the audit. `project_manage action=deps` asks OSV about the version a
52
+ `package-lock.json` beside a `package.json` resolved when it satisfies the range (a stale lock, or a dependency missing
53
+ from it, keeps the lower bound of the range). The setup audit runs the same check for every registered project and
54
+ reports a warning per manifest with advisories, or a note when OSV cannot answer; the 6-hour advisory cache keeps the
55
+ 30-minute schedule from asking again. `read_document` and the index read `.xlsm` workbooks like `.xlsx` (XML parts
56
+ only; `.xlsb`, `.xls`, `.ppt` and `.pptm` stay unread). Tests: `tests/test_second_lens.py`.
57
+ - `project_manage action=pr_impact` lists the tests that reach each touched function, the functions no test reaches and
58
+ the notes the edit hook would have added to the whole diff, so commits made without the hook get the same checks.
59
+ New actions `hotspots`, `dead_code`, `import_cycles` and `comments` answer the questions the generated pages hold
60
+ without writing them. `layer_mode` (default `warn`, on the setup screen) flags an edit that adds a call from a lower
61
+ layer to a higher one of the confirmed layers. `smart_search` takes `folder` and `layer`, applied before ranking. The
62
+ setup screen shows the 30-day model use over every project and an hourly volume alert (twice the busiest hour of the
63
+ purpose's previous week, at least 100, mutable for 7 days, never a notification). Tests: `tests/test_pr_impact.py`,
64
+ `tests/test_code_health.py`, `tests/test_layer_check.py`, `tests/test_search_scope.py`,
65
+ `tests/test_usage_overview.py`.
66
+ - `smart_scout` answers with the best excerpt of a PDF, docx, pptx or xlsx: the window is chosen from the lines the
67
+ index stored for the file (`indexer.indexed_lines`) instead of falling back to the search snippet, and the document
68
+ is never extracted again. A file the index holds no lines of keeps the snippet. Tests: `tests/test_scout.py`.
69
+
70
+ ## 0.9.79 - beta
71
+
72
+ - A graph cut by its limits says what it left out: `left_out` with the files, symbols and call or import relations
73
+ dropped, and a sentence in the note, in the graph answers (`project_manage action=graph`), `layers`, `docs generate`,
74
+ `duplicates`, the affected tests and the PR impact. Files skipped by the 24 MB analysis text limit count with the
75
+ files over the file limit. The graph cache is rebuilt once. The limits stay as they are: doubling them took the
76
+ build of a 6,133-file project from 23 s to 57 s and its memory from 0.7 GB to 1.4 GB
77
+ (`docs/research/2026-11-15-tetos-do-grafo-medidos.md`). The gateway rerank gain over the hybrid order was measured on
78
+ the gold set (first place 35 to 39 of 43, MRR@5 0.868 to 0.919, 1.4 s median per search), so rerank stays optional
79
+ (`docs/research/2026-11-15-ganho-do-rerank-medido.md`). Tests: `tests/test_code_graph_limits.py`,
80
+ `tests/test_affected_tests.py`, `tests/test_pr_impact.py`.
81
+
3
82
  ## 0.9.78 - beta
4
83
 
5
84
  - A registered project whose folder no longer exists is junk: `project_manage action=junk` lists it and the junk
package/README.md CHANGED
@@ -8,7 +8,7 @@ A local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharpe
8
8
  | `smart_scout` | Answers a question about the code with the lines that answer it: reads the candidate files inside Smart Tool and returns `path:lines` with the literal text, so the files never enter the agent's context; with a `scout_model`, a small model also writes a short answer and keeps only the lines that support it; a question about why something is as it is (why, reason, decided) also lists up to three decision notes from the notes the project keeps about its intent, as ids with one excerpt each |
9
9
  | `web_search` / `web_search_result` | Web search across several sources in parallel, in the query language and in English, with a cache shared across sessions; `depth="research"` runs a multi-round research loop |
10
10
  | `web_fetch` | Reads a known URL and returns only the answer to your prompt, written by a small model; pages are cached for 24 h. Pages over 60,000 characters are not cut: the model reads the parts closest to the prompt (needs an `embedding_model`) |
11
- | `read_document` | Reads a PDF, Word, PowerPoint or Excel file as compact text, a range at a time: type, size, page/slide/sheet count and estimated tokens first, then only the pages, slides or sheets asked for (`range="3-5"`), so the file never enters the agent's context whole. Opt-in: the native Read is not redirected. Scanned PDFs fail with that reason (no OCR) |
11
+ | `read_document` | Reads a PDF, Word, PowerPoint, Excel, HTML, CSV or EPUB file as compact text, a range at a time: type, size, page/slide/sheet/part/chapter count and estimated tokens first, then only the units asked for (`range="3-5"`), so the file never enters the agent's context whole. Opt-in: the native Read is not redirected. Scanned PDFs fail with that reason (no OCR) |
12
12
  | `project_manage` | Registers and indexes projects, shows coverage, code maps, duplicated functions and index storage; `graph` with `symbol` tells what changing a function touches: callers, calls, the tests that reach it and the files that import it |
13
13
 
14
14
  Agents that must click, type or log in also get Microsoft's [Playwright MCP](https://github.com/microsoft/playwright-mcp),
@@ -17,7 +17,7 @@ registered as the `playwright` server where this machine lets a browser be drive
17
17
  An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
18
18
  are cheaper, or just tells the agent what Smart Tool would do.
19
19
 
20
- **Status:** 0.9.78 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
20
+ **Status:** 0.9.80 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
21
21
 
22
22
  ## How it works
23
23
 
@@ -59,7 +59,8 @@ session on the machine, so the same question asked twice costs nothing the secon
59
59
  **Reading a page.** `web_fetch` takes a URL and a question. It downloads the page over plain HTTP; when the page only
60
60
  shows its content with JavaScript, it opens it in a headless browser ([Moli](https://github.com/lexmount/moli)
61
61
  first, Chromium as the fallback), which never reaches this machine or your local network. PDFs are read as text, page
62
- by page. A small model then reads the page and returns only the answer, with the source.
62
+ by page, and so are Word, PowerPoint and Excel files (docx, pptx, xlsx), by content type or by the extension of the URL
63
+ when the server sends a generic binary type. A small model then reads the page and returns only the answer, with the source.
63
64
  The agent receives a paragraph instead of the whole page.
64
65
 
65
66
  **Steering the agent (optional hook).** Agents keep reaching for their built-in tools out of habit. The hook sees
@@ -83,7 +84,7 @@ gateway such as Ollama, nothing about your code leaves the machine.
83
84
  ## Install
84
85
 
85
86
  ```powershell
86
- npx @allansantos-dev/smart-tool@0.9.78 install
87
+ npx @allansantos-dev/smart-tool@0.9.80 install
87
88
  ```
88
89
 
89
90
  Updates install themselves, on by default: the tray checks npm when it starts (at logon) and every 24 h, and when there
@@ -143,10 +144,13 @@ Redirects) and Maintenance (Setup audit, Startup).
143
144
  show again**) and the **New recommended models** card on this page offer them: nothing changes until you apply, the
144
145
  models you had are kept, and **Switch back to my previous models** undoes it. The **Model failures (last 24 h)** table
145
146
  groups the failed gateway calls by type (429, 503, other 5xx, timeout, connection, authentication) and role, with the count, the first and last time and one sample, and shows the calls per purpose; a model that fails half of its calls (5 or more) also raises a
146
- notification and a tray alert. A failure inside a tool call also comes back as a warning line in the tool answer;
147
+ notification and a tray alert. The **Model use (last 30 days)** card adds the calls, failures and, when the gateway
148
+ reports them, tokens and cost per purpose over every project, and flags a purpose whose calls in one hour pass twice
149
+ its own busiest hour of the previous week (and at least 100); the flag has **Mute for 7 days** and never raises a
150
+ notification. A failure inside a tool call also comes back as a warning line in the tool answer;
147
151
  calls made by a hook or by background work (index scope and profile, block-pattern proposals) have no answer to
148
- carry it, so they show only in the table, the log and the notification. The local embedder is not in that table or in
149
- the usage meter: it is a sidecar on your machine with no price, and its state is the **Local models** section.
152
+ carry it, so they show only in the table, the log and the notification. The embeddings home records its calls and failures there like any other model, with no price; its
153
+ state, and the reason when it cannot identify its model, is the **Embedder** section.
150
154
  3. **Agents**: register the MCP server in Claude Code or Codex (runs their official `mcp add` command), install the
151
155
  hook and choose its behavior under **Agent rules**. A hook written by an older Smart Tool shows as outdated, with
152
156
  what an update changes; updating previews the entry and rewrites only Smart Tool's, after a backup. Every
@@ -268,16 +272,38 @@ Redirects) and Maintenance (Setup audit, Startup).
268
272
  `llms.txt`; it never creates the file and rewrites only the block.
269
273
 
270
274
  **Impact of a change**: `project_manage action=pr_impact [base=main]` returns a Markdown report of the change set
271
- against `base` (default `HEAD`): the touched functions with their callers, the documentation anchors left out of
272
- date, and the tests to run with their commands. It reads only; publishing it as a pull-request comment is left to
273
- the CI step that calls it.
274
-
275
- **Dependency check on edit**: an edit of `package.json`, `requirements*.txt` or `go.mod` that adds or changes a dependency lists the OSV advisory IDs of up to 5 of them (exact version, or the lower bound of `^`, `~`, `~=` and `>=`), answered from a 6 h disk cache or one 2.5 s batch call that sends only package names and versions; when OSV cannot answer, the reminder says the check was skipped and the edit goes on. `dependency_mode` in `config.json`: `warn` (default) or `off`.
275
+ against `base` (default `HEAD`): the touched functions with their callers and the tests that reach them (the ones no
276
+ test reaches are listed apart), the documentation anchors left out of date, the decision notes and commits that name
277
+ a touched function (a single-word name counts only as a code reference), the files that usually change in the same
278
+ commits and are missing from the diff, the notes the edit hook would have added to the whole diff (comments, copies,
279
+ dependency advisories, credentials) and the tests to run with their commands. It reads
280
+ only; publishing it as a pull-request comment is left to the CI step that calls it.
281
+
282
+ **Shape of the project**: `project_manage action=hotspots` ranks the code files by commits times lines (the
283
+ riskiest to change), `dead_code` lists the functions no code calls and those only tests call (static call graph,
284
+ candidates to read before deleting), `import_cycles` lists the groups of files that import each other, and
285
+ `comments` lists comments in the wrong place across the code files (commented-out code, history tags, loose docs,
286
+ long inline blocks), for code written before the edit hook existed. Each takes `limit` and writes nothing.
287
+
288
+ **Layers on edit**: when the project has confirmed layers (`.smart-tool/layers.json`), an edit that adds a call from
289
+ a lower layer to a function of a higher one is flagged to the agent; it never blocks. `layer_mode` in
290
+ `config.json`: `warn` (default) or `off`, also on the setup screen.
291
+
292
+ **Folder and layer in a search**: `smart_search folder="<path>"` (or a list) and `layer="<name>"` restrict the
293
+ search to a folder of the project or to one of its confirmed layers, by their path globs. The filter runs before
294
+ the ranking, so the results are the best inside the scope, not a cut of the global list.
295
+
296
+ **Credential check on edit**: the lines an edit adds or changes in any file are checked for credential shapes that cannot be
297
+ anything else (GitHub, Slack, Stripe live, Anthropic and npm tokens, Google API keys, AWS access key ids and secret key lines, a
298
+ private-key header at the start of a line); the note names the file and line, never repeats the value and never blocks, and a
299
+ line that says it is an example is skipped.
300
+
301
+ **Dependency check on edit**: an edit of `package.json`, `requirements*.txt`, `pyproject.toml` (`project.dependencies` and `project.optional-dependencies`; Poetry tables are not read), `go.mod`, `Cargo.toml` (crates.io registry dependencies) or `pom.xml` (Maven, plain or property-resolved versions) that adds or changes a dependency lists the OSV advisory IDs of up to 5 of them (exact version, or the lower bound of `^`, `~`, `~=` and `>=`), answered from a 6 h disk cache or one 2.5 s batch call that sends only package names and versions; when OSV cannot answer, the reminder says the check was skipped and the edit goes on. `dependency_mode` in `config.json`: `warn` (default) or `off`.
276
302
 
277
303
  **PDFs** inside a project are indexed as text with `[page N]` markers (20 MiB, 300 pages and 20 s per file; a scanned, password-protected or corrupt PDF is skipped with the reason), so `smart_search` returns the page that holds a passage. **Presentations and workbooks** (`.pptx`, `.xlsx`) are indexed the
278
304
  same way: text of each slide under `[slide N]` with the speaker notes under `[notes N]`, each sheet under `[sheet NAME]` with one
279
- tab-separated line per row (cached formula values; 100,000 rows or 1 MiB of text at most). Older formats (`.ppt`, `.xls`,
280
- `.xlsm`) are not read.
305
+ tab-separated line per row (cached formula values; 100,000 rows or 1 MiB of text at most). A macro-enabled `.xlsm` is read
306
+ like an `.xlsx` (its XML parts only, no macro runs). Older formats (`.ppt`, `.xls`, `.xlsb`, `.pptm`) are not read.
281
307
 
282
308
  **Block patterns waiting for you** are shown in one approval panel: 8 per page, the ones that would have blocked
283
309
  the most logged calls first, a filter over pattern, reason and example, and bulk accept or reject of the selected
@@ -306,6 +332,17 @@ Redirects) and Maintenance (Setup audit, Startup).
306
332
  unregisters them and deletes their index files (never the project's own files) and writes a manifest in
307
333
  `junk-manifests`. The daemon applies the same test at start.
308
334
 
335
+ `project_manage action=deps project_root="<project>"` lists the known OSV advisories of every dependency the manifests
336
+ of the project declare (`package.json`, `requirements*.txt`, `pyproject.toml`, `go.mod`, `Cargo.toml`, `pom.xml`), not only the ones an edit changed;
337
+ a `package.json` with a `package-lock.json` beside it is checked at the locked version when it satisfies the range. Only package
338
+ names and versions leave the machine, in batches of 100 per ecosystem. The setup audit runs the same check for every
339
+ registered project (a warning per manifest with advisories; the 6 h cache keeps it from asking OSV on every run).
340
+
341
+ `smart_search mode=dossier history=true` also searches the messages of the commits that have a body, and
342
+ `smart_search decision_notes=true` appends the decision notes of the project (research notes, backlog, CLAUDE.md,
343
+ AGENTS.md and commit messages) to a search answer; both are off by default, and the first read of a repository's
344
+ history can take longer than the dossier budget, which the answer then says.
345
+
309
346
  `project_manage` with `action=setup_audit` and a `project_root` (or `python setup_audit.py --project <folder>`)
310
347
  reports what Claude Code and Codex actually load for that folder (settings layers, enabled plugins of both,
311
348
  hooks, MCP servers, skills, CLAUDE.md and AGENTS.md) and where they collide: several hooks deciding on the same tool in
@@ -335,10 +372,20 @@ Redirects) and Maintenance (Setup audit, Startup).
335
372
  definition in `~/.codex/config.toml`, as `/hooks` would, so no `/hooks` step is left; an entry Codex does not trust
336
373
  shows "Not approved in Codex", and **Update** records it. It is off by default. The hook script
337
374
  (`hooks/session_audit.py`) only asks the running service and prints nothing when the service is down.
338
- 4. **Local models** (optional): if the mcp-memory embedding sidecar is installed on the machine, its local embedder
339
- can serve as a fallback (or preferred) embedding model.
375
+ 4. **Embedder**: the embeddings home (casa-de-embeddings, the embedding service of this machine, found through
376
+ `~/.casa-de-embeddings/run/daemon.json`) is the default; the gateway's `embedding_model` and `rerank_model` are used
377
+ only when you choose **Gateway** here, never in place of a stopped home. The home's embedder is identified by the
378
+ `model_sha256` and `dimension` of its `/info`; a home that answers without them is reported as an error, never
379
+ replaced silently by lexical search. Rerank goes through the home's connector to the rerank host configured in it;
380
+ without one, results keep the hybrid order and say so. Switching the embedder re-embeds each project in the
381
+ background, one at a time: the old index keeps answering, the folder switches to the new index when it is complete,
382
+ and the vectors of the old model are then deleted. While the home is not running, a folder indexed with another model
383
+ keeps answering from that index and is not updated.
340
384
  5. **Web search providers** work without keys on their free public tiers; add your own key for more volume. Set your
341
- own contact (e-mail or URL) if you want public APIs such as Wikimedia to see it in the User-Agent.
385
+ own contact (e-mail or URL) if you want public APIs such as Wikimedia to see it in the User-Agent. The same page lists
386
+ who asked the recent searches (client and project folder, never the text of a search).
387
+ 6. **Automatic updates** (Startup page): the switch the tray menu also has, with the installed version, the last npm check,
388
+ the last update and a failed automatic try; no network call is made to draw it.
342
389
 
343
390
  ## Writing a model adapter
344
391
 
@@ -390,7 +437,7 @@ fails to load is listed in the setup screen with its error and does not affect t
390
437
  ## Uninstall
391
438
 
392
439
  ```powershell
393
- npx @allansantos-dev/smart-tool@0.9.78 uninstall [--target DIR] [--keep-data]
440
+ npx @allansantos-dev/smart-tool@0.9.80 uninstall [--target DIR] [--keep-data]
394
441
  ```
395
442
 
396
443
  It asks nothing and opens no window. It stops the tray and the daemon, removes the logon task, the hook entries of Smart
package/affected_tests.py CHANGED
@@ -413,6 +413,10 @@ def affected(root, base="HEAD", limit=30, view_id=None):
413
413
  "view": (data.get("selected") or {}).get("label"),
414
414
  "note": NOTE,
415
415
  }
416
+ if data.get("truncated"):
417
+ result["truncated"] = True
418
+ result["left_out"] = data["left_out"]
419
+ result["note"] += " " + code_graph.left_out_sentence(data) + " Tests beyond the cut are not listed."
416
420
  if len(ordered) > limit:
417
421
  result["omitted"] = {"tests": len(ordered) - limit}
418
422
  if not changes:
@@ -29,6 +29,7 @@ import time
29
29
  import tomllib
30
30
 
31
31
  import atomic_io
32
+ import chromium_flags
32
33
  import codex_config
33
34
  import endpoint_sync
34
35
  import install_runtime
@@ -44,14 +45,19 @@ CONFIG_PATH = os.path.join(paths.DATA_DIR, "playwright-mcp.json")
44
45
  HANDSHAKE_PATH = os.path.join(paths.DATA_DIR, "playwright-mcp-handshake.json")
45
46
  PROXY = os.path.join(os.path.dirname(os.path.abspath(__file__)), "lazy_mcp_proxy.py")
46
47
  AGENT_UA_MARK = "SmartToolAgentBrowser"
48
+ MCP_PLAYWRIGHT_BUNDLE = install_runtime.NODE_DIR / "node_modules" / "playwright-core" / "lib" / "coreBundle.js"
47
49
  OUTPUT_MAX_BYTES = 100 * 1024 * 1024
48
50
  PROBE_TIMEOUT_S = 60
49
51
  DENIED_TOOLS = ("browser_run_code_unsafe",)
50
52
  _CODEX_TABLE_RE = re.compile(rf"^\[mcp_servers\.{SERVER}\][ \t]*\r?\n", re.M)
51
- _CHROMIUM_PROBE = ("from playwright.sync_api import sync_playwright\n"
53
+ _CHROMIUM_PROBE = ("import sys\n"
54
+ f"sys.path.insert(0, {os.path.dirname(os.path.abspath(__file__))!r})\n"
55
+ "import chromium_flags\n"
56
+ "from playwright.sync_api import sync_playwright\n"
52
57
  "with sync_playwright() as p:\n"
53
58
  " print(p.chromium.executable_path)\n"
54
- " b = p.chromium.launch(headless=True); print(b.new_page().evaluate('navigator.userAgent')); b.close()")
59
+ " b = p.chromium.launch(headless=True, args=chromium_flags.python_chromium_args())\n"
60
+ " print(b.new_page().evaluate('navigator.userAgent')); b.close()")
55
61
 
56
62
 
57
63
  def chromium():
@@ -100,7 +106,8 @@ def write_config(user_agent):
100
106
  a CORS request does not count it as an author header, so no preflight is added; a header in extraHTTPHeaders
101
107
  forced one on every cross-origin fetch and font."""
102
108
  os.makedirs(os.path.dirname(CONFIG_PATH), exist_ok=True)
103
- config = {"browser": {"contextOptions": {"userAgent": f"{user_agent} {AGENT_UA_MARK}"}}}
109
+ config = {"browser": {"launchOptions": {"args": [chromium_flags.disable_features_arg(MCP_PLAYWRIGHT_BUNDLE)]},
110
+ "contextOptions": {"userAgent": f"{user_agent} {AGENT_UA_MARK}"}}}
104
111
  atomic_io.write_secret_text(CONFIG_PATH, json.dumps(config, indent=1))
105
112
  return CONFIG_PATH
106
113
 
@@ -8,6 +8,8 @@ import threading
8
8
  import time
9
9
  import uuid
10
10
 
11
+ import process_limits
12
+
11
13
  START_TIMEOUT_S = 45
12
14
  START_FAILURE_COOLDOWN_S = 60
13
15
  SERVICE_IDLE_S = 300
@@ -28,37 +30,6 @@ class BrowserExited(RuntimeError):
28
30
  pass
29
31
 
30
32
 
31
- def _kill_on_close_job():
32
- import ctypes
33
- from ctypes import wintypes
34
-
35
- class IoCounters(ctypes.Structure):
36
- _fields_ = [(name, ctypes.c_ulonglong) for name in
37
- ("read_ops", "write_ops", "other_ops", "read_bytes", "write_bytes", "other_bytes")]
38
-
39
- class BasicLimits(ctypes.Structure):
40
- _fields_ = [("per_process_time", ctypes.c_longlong), ("per_job_time", ctypes.c_longlong),
41
- ("flags", wintypes.DWORD), ("min_ws", ctypes.c_size_t), ("max_ws", ctypes.c_size_t),
42
- ("active_processes", wintypes.DWORD), ("affinity", ctypes.c_size_t),
43
- ("priority", wintypes.DWORD), ("scheduling", wintypes.DWORD)]
44
-
45
- class ExtendedLimits(ctypes.Structure):
46
- _fields_ = [("basic", BasicLimits), ("io", IoCounters), ("process_memory", ctypes.c_size_t),
47
- ("job_memory", ctypes.c_size_t), ("peak_process_memory", ctypes.c_size_t),
48
- ("peak_job_memory", ctypes.c_size_t)]
49
-
50
- kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
51
- kernel32.CreateJobObjectW.restype = wintypes.HANDLE
52
- job = kernel32.CreateJobObjectW(None, None)
53
- if not job:
54
- raise ctypes.WinError(ctypes.get_last_error())
55
- limits = ExtendedLimits()
56
- limits.basic.flags = 0x2000 # JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
57
- if not kernel32.SetInformationJobObject(wintypes.HANDLE(job), 9, ctypes.byref(limits), ctypes.sizeof(limits)):
58
- raise ctypes.WinError(ctypes.get_last_error())
59
- return kernel32, job
60
-
61
-
62
33
  class BrowserService:
63
34
  def __init__(self, executable, script):
64
35
  self.executable = executable
@@ -126,7 +97,7 @@ class BrowserService:
126
97
  # Kill-on-close job: the browser tree dies with the daemon even when the daemon is killed hard.
127
98
  try:
128
99
  if self.job is None:
129
- self.job = _kill_on_close_job()
100
+ self.job = process_limits.kill_on_close_job()
130
101
  kernel32, job = self.job
131
102
  import ctypes
132
103
  from ctypes import wintypes
@@ -0,0 +1,30 @@
1
+ """Chromium switches carried by every automation browser launch of Smart Tool.
2
+
3
+ `WebUsbDeviceDetection` makes Chromium enumerate USB devices a few seconds after launch, which freezes a USB
4
+ microphone that is recording. Chromium keeps only the last `--disable-features` switch, and Playwright puts its own
5
+ list first and appends the caller's arguments, so a lone switch would drop Playwright's list: the switch built here
6
+ repeats the list of the Playwright bundle the browser is launched by and adds the feature to it.
7
+ """
8
+ import re
9
+ from pathlib import Path
10
+
11
+ WEB_USB_FEATURE = "WebUsbDeviceDetection"
12
+ _FEATURE_LIST = re.compile(r"\bdisabledFeatures = \[(.*?)\]\.filter\(", re.S)
13
+ _FEATURE_NAME = re.compile(r'^\s*"([A-Za-z0-9]+)"', re.M)
14
+
15
+
16
+ def disable_features_arg(bundle):
17
+ """The single `--disable-features` switch for a launch through the Playwright driver bundle `bundle`
18
+ (coreBundle.js): that bundle's own list plus WebUsbDeviceDetection. RuntimeError when the list is not found, so a
19
+ Playwright upgrade that moves it fails loudly instead of launching without the switch."""
20
+ found = _FEATURE_LIST.search(Path(bundle).read_text(encoding="utf-8"))
21
+ names = _FEATURE_NAME.findall(found.group(1)) if found else []
22
+ if not names:
23
+ raise RuntimeError(f"No disabledFeatures list in {bundle}; chromium_flags must follow this Playwright version.")
24
+ return "--disable-features=" + ",".join([*names, WEB_USB_FEATURE])
25
+
26
+
27
+ def python_chromium_args():
28
+ """Launch arguments for Chromium started by the Python Playwright package that is imported here."""
29
+ import playwright
30
+ return [disable_features_arg(Path(playwright.__file__).parent / "driver" / "package" / "lib" / "coreBundle.js")]
package/code_graph.py CHANGED
@@ -23,7 +23,7 @@ import web_document_graph
23
23
  import code_graph_go
24
24
  import document_text
25
25
 
26
- VERSION = 5
26
+ VERSION = 6
27
27
  SOURCE_EXTENSIONS = ('.py','.go','.mod','.java','.js','.jsx','.ts','.tsx','.mjs','.cjs','.mts','.cts','.json','.html','.htm','.css','.scss','.sass','.less','.md','.markdown')
28
28
  MAX_OVERLAY_BYTES = 5 * 1024 * 1024
29
29
  MAX_FILES = 3000
@@ -71,9 +71,10 @@ def _snapshot(path):
71
71
  columns = {row[1] for row in conn.execute('PRAGMA table_info(manifest)')}
72
72
  size = 'm.size' if 'size' in columns else 'NULL'
73
73
  mtime = 'm.mtime_ns' if 'mtime_ns' in columns else 'NULL'
74
- records = conn.execute(f'SELECT m.path,m.hash,{size},{mtime},count(c.id),max(c.end_line) FROM manifest m LEFT JOIN chunks c ON c.path=m.path GROUP BY m.path ORDER BY m.path LIMIT ?', (MAX_FILES+1,)).fetchall()
75
- files, sources, diagnostics, used = [], {}, [], 0
76
- for original, digest, byte_size, stamp, chunks, line_count in records[:MAX_FILES]:
74
+ records = conn.execute(f'SELECT m.path,m.hash,{size},{mtime},count(c.id),max(c.end_line) FROM manifest m LEFT JOIN chunks c ON c.path=m.path GROUP BY m.path ORDER BY m.path LIMIT ?', (MAX_FILES,)).fetchall()
75
+ total = conn.execute('SELECT count(*) FROM manifest').fetchone()[0]
76
+ files, sources, diagnostics, used, over_text = [], {}, [], 0, 0
77
+ for original, digest, byte_size, stamp, chunks, line_count in records:
77
78
  file_path = original.replace('\\','/')
78
79
  record = {'id':file_path,'path':file_path,'hash':digest,'size':byte_size,'mtime_ns':stamp,
79
80
  'chunks':chunks,'lines':line_count or 0,'group':posixpath.dirname(file_path) or '(root)',
@@ -87,10 +88,11 @@ def _snapshot(path):
87
88
  diagnostics.append({'path':file_path,'reason':error})
88
89
  elif used+len(text) > MAX_TEXT:
89
90
  diagnostics.append({'path':file_path,'reason':'Content limit for this analysis reached.'})
91
+ over_text += 1
90
92
  else:
91
93
  used += len(text)
92
94
  sources[file_path] = text
93
- return files, sources, diagnostics, len(records)>MAX_FILES
95
+ return files, sources, diagnostics, max(total-MAX_FILES, 0) + over_text
94
96
  finally:
95
97
  conn.close()
96
98
 
@@ -247,7 +249,8 @@ def _python_graph(sources):
247
249
  visit_scope(tree)
248
250
  if used_module:symbols.append(module)
249
251
  return {'symbols':symbols,'dependencies':dependencies[:MAX_EDGES],'calls':calls,'unresolved':unresolved,
250
- 'diagnostics':diagnostics,'parsed':list(trees),'truncated':any(dropped.values()) or len(dependencies)>MAX_EDGES}
252
+ 'diagnostics':diagnostics,'parsed':list(trees),
253
+ 'left_out':{'symbols':dropped['symbols'],'edges':dropped['calls']+max(len(dependencies)-MAX_EDGES,0)}}
251
254
 
252
255
 
253
256
  NODE_HEAP_MB = (1024, 4096)
@@ -298,15 +301,22 @@ def _analyze(path):
298
301
  return _analyze_sources(*_snapshot(path))
299
302
 
300
303
 
301
- def _analyze_sources(files,sources,diagnostics,truncated):
304
+ def _analyze_sources(files,sources,diagnostics,files_left_out):
305
+ """Analyzes the sources of one snapshot. `left_out` counts what the limits cut: files the snapshot never read
306
+ (files_left_out), symbols and call or import occurrences each analyzer dropped at its own cap, and what the merged
307
+ graph drops at the global caps (symbols past MAX_SYMBOLS, calls whose endpoint was dropped, relations past MAX_EDGES).
308
+ The counts are lower bounds: what lies inside left-out files, and the calls made inside functions an analyzer
309
+ dropped, are never seen and not counted."""
302
310
  merged={'symbols':[],'dependencies':[],'calls':[],'unresolved':[],'diagnostics':diagnostics,'parsed':[]}
311
+ left_out={'files':files_left_out,'symbols':0,'edges':0}
303
312
  retryable=False
304
313
  for output in (_python_graph(sources),_javascript_graph(sources,[f['path'] for f in files]),_java_graph(sources),code_graph_go.analyze(sources,ANALYZER_LIMITS),web_document_graph.analyze(sources,[f['path'] for f in files])):
305
314
  for key in merged:merged[key].extend(output.get(key,[]))
306
- truncated |= output.get('truncated',False)
315
+ for kind,count in output.get('left_out',{}).items():left_out[kind]+=count
307
316
  retryable |= output.get('retryable',False)
308
- truncated |= len(merged['symbols'])>MAX_SYMBOLS
309
- by_id={sym['id']:sym for sym in merged['symbols'][:MAX_SYMBOLS]}
317
+ kept_symbols=merged['symbols'][:MAX_SYMBOLS]
318
+ left_out['symbols']+=len(merged['symbols'])-len(kept_symbols)
319
+ by_id={sym['id']:sym for sym in kept_symbols}
310
320
  # Relações repetidas preservam contagem e locais de chamada sem poluir o desenho.
311
321
  def collapse(rows,fields):
312
322
  result={}
@@ -316,14 +326,17 @@ def _analyze_sources(files,sources,diagnostics,truncated):
316
326
  result[key]['count']+=1
317
327
  if len(result[key]['lines'])<20:result[key]['lines'].append(row.get('line'))
318
328
  return list(result.values())
329
+ linked=[r for r in merged['calls'] if r['source'] in by_id and r['target'] in by_id]
330
+ left_out['edges']+=len(merged['calls'])-len(linked)
319
331
  deps=collapse(merged['dependencies'],('source','target','specifier','kind'))
320
- calls=collapse([r for r in merged['calls'] if r['source'] in by_id and r['target'] in by_id],('source','target','kind'))
332
+ calls=collapse(linked,('source','target','kind'))
333
+ left_out['edges']+=sum(r['count'] for r in deps[MAX_EDGES:])+sum(r['count'] for r in calls[MAX_EDGES:])
321
334
  supported=set(merged['parsed'])
322
335
  for file in files:file['analysis']='parsed' if file['path'] in supported else 'not_analyzed'
323
- return {'files':files,'symbols':merged['symbols'][:MAX_SYMBOLS],
336
+ return {'files':files,'symbols':kept_symbols,
324
337
  'dependencies':deps[:MAX_EDGES],'calls':calls[:MAX_EDGES],
325
338
  'unresolved':merged['unresolved'][:1000],'diagnostics':merged['diagnostics'],
326
- 'truncated':truncated or len(deps)>MAX_EDGES or len(calls)>MAX_EDGES,'retryable':retryable,
339
+ 'truncated':any(left_out.values()),'left_out':left_out,'retryable':retryable,
327
340
  'counts':{'files':len(files),'parsed_files':len(supported),'symbols':len(merged['symbols']),
328
341
  'resolved_imports':sum(bool(r['target']) for r in deps),
329
342
  'external_imports':sum(r['resolution']=='external' for r in deps),
@@ -353,13 +366,21 @@ def _focus(data,file_path):
353
366
  return data
354
367
 
355
368
 
369
+ def left_out_sentence(data):
370
+ """One sentence with how much the analysis limits cut from the graph, or None when nothing was left out."""
371
+ if not data.get('truncated'):
372
+ return None
373
+ left=data['left_out']
374
+ return (f"Graph cut by analysis limits: {left['files']:,} files, {left['symbols']:,} symbols and {left['edges']:,} call and import relations left out"
375
+ +' (lower bounds: what lies inside left-out files and functions is not counted).')
376
+
377
+
356
378
  def _by_file(data):
357
379
  by_file={}
358
380
  for symbol in data['symbols']:
359
381
  by_file.setdefault(symbol['path'],[]).append(symbol)
360
382
  parsed={f['path'] for f in data['files'] if f.get('analysis')=='parsed'}
361
- note='Graph truncated by analysis limits; some files have no functions.' if data.get('truncated') else None
362
- return {'symbols':by_file,'parsed':parsed,'note':note}
383
+ return {'symbols':by_file,'parsed':parsed,'note':left_out_sentence(data)}
363
384
 
364
385
 
365
386
  def _disk_path(view_path):
@@ -493,6 +514,7 @@ def build(root,view_id=None,storage_id=None,file_path=None,background=False):
493
514
  file['status']='indexed' if (current.st_mtime_ns,current.st_size)==(file['mtime_ns'],file['size']) else 'pending'
494
515
  except OSError:file['status']='missing'
495
516
  file.pop('mtime_ns',None)
517
+ if data['truncated']:data['note']=left_out_sentence(data)
496
518
  data.update(views=inventory['views'],selected=inventory['selected'],current_view=inventory['current_view'],
497
519
  message=inventory['message'],read_only=True,source='indexed_snapshot',generated_at=time.time(),
498
520
  limits={'files':MAX_FILES,'symbols':MAX_SYMBOLS,'edges':MAX_EDGES,'display_nodes_default':120},
@@ -601,7 +623,7 @@ def build_overlay(root,changes,view_id=None,shared=False):
601
623
  if not inventory['selected']:
602
624
  return {**inventory,'symbols':[],'dependencies':[],'calls':[],'diagnostics':[],'counts':{},'files':[]}
603
625
  path=inventory['selected']['path'];stat=os.stat(path)
604
- files,sources,diagnostics,truncated=_snapshot(path)
626
+ files,sources,diagnostics,files_left_out=_snapshot(path)
605
627
  by_path={f['path']:f for f in files};overlaid=[]
606
628
  for rel,status in sorted(changes.items()):
607
629
  full=os.path.join(root,rel)
@@ -624,7 +646,7 @@ def build_overlay(root,changes,view_id=None,shared=False):
624
646
  if not _ANALYSIS_SLOTS.acquire(timeout=3):
625
647
  raise RuntimeError('Two map analyses are in progress. Wait for them to finish and retry.')
626
648
  try:
627
- cached=_analyze_sources(list(by_path.values()),sources,diagnostics,truncated)
649
+ cached=_analyze_sources(list(by_path.values()),sources,diagnostics,files_left_out)
628
650
  finally:
629
651
  _ANALYSIS_SLOTS.release()
630
652
  if not cached['retryable']:
package/code_graph_go.py CHANGED
@@ -352,7 +352,7 @@ def _default_alias(spec):
352
352
 
353
353
  def analyze(sources, limits=None):
354
354
  """Imports, symbols (functions, methods, types) and resolved calls of the Go files in sources (path -> text).
355
- limits caps symbols and edges ({'symbols': n, 'edges': n}); anything left out sets 'truncated'."""
355
+ limits caps symbols and edges ({'symbols': n, 'edges': n}); 'left_out' counts the symbols and calls dropped at the caps."""
356
356
  limits = limits or {}
357
357
  max_symbols, max_edges = limits.get("symbols", float("inf")), limits.get("edges", float("inf"))
358
358
  go_files = {path: text for path, text in sources.items() if path.endswith(".go")}
@@ -428,7 +428,7 @@ def _symbols_and_calls(files, max_symbols, max_edges):
428
428
  target = imports_of[path].get(alias)
429
429
  return (target, name) if target else None
430
430
 
431
- index, results, symbols, calls, unresolved, truncated = {}, {}, [], [], [], False
431
+ index, results, symbols, calls, unresolved, left_out = {}, {}, [], [], [], {"symbols": 0, "edges": 0}
432
432
  for path, info in files.items():
433
433
  tables = index.setdefault(key_of(path), {"funcs": {}, "methods": {}, "types": {}, "fields": {},
434
434
  "embedded": {}})
@@ -449,7 +449,7 @@ def _symbols_and_calls(files, max_symbols, max_edges):
449
449
  symbol_id = f"{symbol_id}@{decl['start']}"
450
450
  seen.add(symbol_id)
451
451
  if len(symbols) >= max_symbols:
452
- truncated = True
452
+ left_out["symbols"] += 1
453
453
  continue
454
454
  symbol = {"id": symbol_id, "path": path, "name": name, "kind": decl["kind"], "language": "go",
455
455
  "start_line": decl["start"], "end_line": decl["end"]}
@@ -484,7 +484,7 @@ def _symbols_and_calls(files, max_symbols, max_edges):
484
484
  calls.append({"source": decl["symbol"]["id"], "target": call["target"]["id"],
485
485
  "line": call["line"], "kind": "reference", "resolution": "static"})
486
486
  else:
487
- truncated = True
487
+ left_out["edges"] += 1
488
488
  continue
489
489
  if call["target"] is None:
490
490
  if len(unresolved) < 1000:
@@ -494,8 +494,8 @@ def _symbols_and_calls(files, max_symbols, max_edges):
494
494
  calls.append({"source": decl["symbol"]["id"], "target": call["target"]["id"],
495
495
  "line": call["line"], "kind": "call", "resolution": "static"})
496
496
  else:
497
- truncated = True
498
- return {"symbols": symbols, "calls": calls, "unresolved": unresolved, "truncated": truncated}
497
+ left_out["edges"] += 1
498
+ return {"symbols": symbols, "calls": calls, "unresolved": unresolved, "left_out": left_out}
499
499
 
500
500
 
501
501
  def _body_calls(path, tokens, decl, index, imported, own, results, resolve_type, member):