@allansantos-dev/smart-tool 0.9.38 → 0.9.40

package/CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## 0.9.40 - beta
+
+- PDFs of the project are searchable. `smart_search` indexes the text layer of a `.pdf` inside the project scope and
+  returns the page (`[page N]`) with the passage. The extractor is the one `web_fetch` already used, now in
+  `document_text`: `pypdf` in an isolated subprocess killed after 20 s, at most 20 MiB, 300 pages and 1,000,000
+  characters per file. A scanned PDF (no text layer), a password-protected one, a corrupt one or one that overruns the
+  deadline is skipped with the reason in the index stats and the rest of the project is indexed as usual. The secret,
+  minified and long-line guards apply to the extracted text. `document_text.VERSION` is 2: every `.docx` already indexed
+  is read once more. Tests in `tests/test_pdf_index.py`; measurement before and after in
+  `docs/research/data/pdf-probe-*.json`.
+- Editing `package.json` warns about known-vulnerable dependencies. Added or changed entries of `dependencies`,
+  `devDependencies`, `peerDependencies` and `optionalDependencies` (at most 5 per edit) are checked against OSV with one
+  `querybatch` call (2.5 s timeout, answers cached on disk for 6 h); the edit reminder lists the advisory IDs per
+  `package@version`, or says `dependency check skipped: <reason>` when OSV cannot be reached, and the edit goes on.
+  Only package names and versions leave the machine. `dependency_mode` in `config.json` (`warn` by default, `off`)
+  turns it off. Tests in `tests/test_dependency_notice.py`; measurement in `docs/research/data/dependency-probe-*.json`.
+- A scheduled setup audit that finds new errors or warnings raises one Windows toast per project with an "Open the
+  audit" button; the first audit, manual runs and info findings stay silent (`tests/test_audit_toast.py`).
+- `smart_scout` answers a question about intent (why, reason, decided) with up to three decision notes from the project
+  notes next to the code lines (`tests/test_scout.py`).
+- The edit reminder gives the command that runs the tests calling the edited function (`tests/test_edit_review_cache.py`).
+
+## 0.9.39 - beta
+
+- Two-letter technical terms (`ui`, `db`, `id`) count again in the shared `focus_terms`, used by `smart_search
+  mode=dossier` and `smart_scout`. Two-letter English and Portuguese function words (`is`, `of`, `de`, `em`, ...) are
+  dropped like the question words. Tests in `tests/test_dossier.py`.
+- The npm registry and OSV checks of `freshness` no longer raise. `npm_registry_evidence` read a local `version` before
+  assigning it (UnboundLocalError on every call) and `npm_advisories` shadowed the `version` module with its parameter;
+  both return evidence or `None` as documented. Tests in `tests/test_npm_evidence.py`.
+
 ## 0.9.38 - beta
 
 - One definition of a search term. `smart_search mode=dossier` now takes its terms from `dossier.focus_terms`, the

package/README.md

@@ -5,7 +5,7 @@ A local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharpe
 | Tool | What it does |
 |---|---|
 | `smart_search` / `smart_search_result` | Semantic + lexical search over an indexed project, results split into code, tests and docs, with the likeliest functions per chunk |
-| `smart_scout` | Answers a question about the code with the lines that answer it: reads the candidate files inside Smart Tool and returns `path:lines` with the literal text, so the files never enter the agent's context; with a `scout_model`, a small model also writes a short answer and keeps only the lines that support it |
+| `smart_scout` | Answers a question about the code with the lines that answer it: reads the candidate files inside Smart Tool and returns `path:lines` with the literal text, so the files never enter the agent's context; with a `scout_model`, a small model also writes a short answer and keeps only the lines that support it; a question about why something is as it is (why, reason, decided) also lists up to three decision notes from the notes the project keeps about its intent, as ids with one excerpt each |
 | `web_search` / `web_search_result` | Web search across several sources in parallel, in the query language and in English, with a cache shared across sessions; `depth="research"` runs a multi-round research loop |
 | `web_fetch` | Reads a known URL and returns only the answer to your prompt, written by a small model; pages are cached for 24 h. Pages over 60,000 characters are not cut: the model reads the parts closest to the prompt (needs an `embedding_model`) |
 | `project_manage` | Registers and indexes projects, shows coverage, code maps, duplicated functions and index storage; `graph` with `symbol` tells what changing a function touches: callers, calls, the tests that reach it and the files that import it |
@@ -16,7 +16,7 @@ registered as the `playwright` server where this machine lets a browser be drive
 An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
 are cheaper, or just tells the agent what Smart Tool would do.
 
-**Status:** 0.9.38 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
+**Status:** 0.9.40 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
 
 ## How it works
 
@@ -181,13 +181,18 @@ Redirects) and Maintenance (Setup audit, Startup).
    `@override`) functions are exempt, as in pydocstyle, eslint-plugin-jsdoc and Checkstyle, and a JSDoc above a set of
    TypeScript overloads documents the implementation:
    - **Remind** (default): the edit runs and the agent is told which touched functions have no docstring, and which
-     documented ones changed their signature, with their callers.
+     documented ones changed their signature, with their callers, the tests calling them and the command that runs those tests
+  (from the code graph, only when it is already cached: the hook never waits for an analysis).
    - **Require**: an edit that leaves a touched function without a docstring is blocked until it adds one.
    - **Off**: no checks.
 
    A documented function edited only in its body is not mentioned. `project_manage` with `action=docs` lists the
    public functions still without a docstring, for documenting a project that started without them.
 
+   **Dependency check on edit**: an edit of `package.json` that adds or changes a dependency lists the OSV advisory IDs of up to 5 of them (exact version, or the lower bound of `^`, `~` and `>=`), answered from a 6 h disk cache or one 2.5 s batch call that sends only package names and versions; when OSV cannot answer, the reminder says the check was skipped and the edit goes on. `dependency_mode` in `config.json`: `warn` (default) or `off`.
+
+   **PDFs** inside a project are indexed as text with `[page N]` markers (20 MiB, 300 pages and 20 s per file; a scanned, password-protected or corrupt PDF is skipped with the reason), so `smart_search` returns the page that holds a passage.
+
    **Block patterns waiting for you** are shown in one approval panel: 8 per page, the ones that would have blocked
    the most logged calls first, a filter over pattern, reason and example, and bulk accept or reject of the selected
    rows; each row opens to its reason and example.
@@ -219,7 +224,9 @@ Redirects) and Maintenance (Setup audit, Startup).
    two minutes after it starts and then every 30 minutes (MCP servers shared by several projects are contacted once
    and their result reused for 6 hours; Run now always contacts them again), and the setup screen's **Setup audit**
    panel shows each project's last report: findings by severity, what is new, what was resolved, the tokens each MCP
-   server's tools cost and what was not verified.
+   server's tools cost and what was not verified. When a scheduled run finds new errors or warnings, a Windows
+   notification from the tray names the folder and up to three of them, once per finding, with a button that opens the
+   panel; manual runs and info findings stay silent.
 
    The agent hears about it without asking for it. `setup_audit` with `mode=last` returns the stored report for the
    folder (or the nearest audited parent) without auditing again, and fails with what to do when none is stored. On

package/config.py

@@ -37,6 +37,7 @@ DEFAULT_CONFIG = {
     "hook_mode": "redirect",
     "doc_mode": "remind",
     "duplicate_mode": "warn",
+    "dependency_mode": "warn",
     "auto_update": True,
     "pattern_proposals": "on",
     "contact": "",
@@ -61,6 +62,9 @@ DOC_MODES = ("require", "remind", "off")
 # Edit hook on functions that copy one already indexed (exact or near-identical body): warn = let the edit run and
 # point to the existing function; off = say nothing. Never blocks: near matches are leads, not defects.
 DUPLICATE_MODES = ("warn", "off")
+# Edit hook on package.json: warn = let the edit run and list the added or changed dependencies that have OSV advisories;
+# off = say nothing and make no network call.
+DEPENDENCY_MODES = ("warn", "off")
 # Router model reviewing, in the background, searches the deterministic redirect rule let run: on = propose block
 # patterns for the user to accept in the setup screen; off = no review.
 PATTERN_PROPOSAL_MODES = ("on", "off")
@@ -85,6 +89,13 @@ def duplicate_mode(cfg=None):
     return value
 
 
+def dependency_mode(cfg=None):
+    value = (cfg if cfg is not None else load_config()).get("dependency_mode") or DEFAULT_CONFIG["dependency_mode"]
+    if value not in DEPENDENCY_MODES:
+        raise ValueError(f"Invalid dependency_mode ({value!r}) in {CONFIG_PATH}: use warn or off.")
+    return value
+
+
 def pattern_proposals(cfg=None):
     """on: the router model reviews searches the redirect rule let run and proposes block patterns for the user to
     accept; off: no review."""

package/dependency_notice.py

@@ -0,0 +1,176 @@
+"""Vulnerable-dependency notice for an edit of package.json, built before the edit runs.
+
+Compares the dependency tables of the file before and after the edit, asks OSV (one `querybatch` call, short timeout)
+about the added or changed ones only, and answers from a disk cache of (package, version) -> advisory IDs for 6 hours.
+Only package names and versions leave the machine. The caller (`hook_decision`) turns a `DependencyCheckError` into
+"dependency check skipped: <reason>" and lets the edit go on.
+"""
+import json
+import os
+import re
+import time
+
+import atomic_io
+import freshness
+import paths
+
+CACHE_PATH = os.path.join(paths.DATA_DIR, "dependency_advisories.json")
+CACHE_TTL_S = 6 * 3600
+OSV_TIMEOUT_S = 2.5
+MAX_PACKAGES = 5
+MAX_IDS_SHOWN = 5
+TABLES = ("dependencies", "devDependencies", "peerDependencies", "optionalDependencies")
+PREFIX = "Smart Tool dependencies: "
+_SPECIFIER_RE = re.compile(r"(\^|~|>=|>|=)?\s*v?(\d+)(?:\.(\d+))?(?:\.(\d+))?"
+                           r"((?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?)")
+
+
+class DependencyCheckError(RuntimeError):
+    """The check could not run (OSV unreachable or answering badly, unreadable cache, current file not JSON)."""
+
+
+def _tables(text):
+    """The (name, specifier) pairs of every dependency table in a package.json text.
+
+    Parameters: text (str) file content.
+    Returns: dict (name, specifier) -> None, in file order; None when the text is not a JSON object.
+    """
+    try:
+        data = json.loads(text)
+    except ValueError:
+        return None
+    if not isinstance(data, dict):
+        return None
+    pairs = {}
+    for table in TABLES:
+        entries = data.get(table)
+        if isinstance(entries, dict):
+            pairs.update({(name, spec): None for name, spec in entries.items() if isinstance(spec, str)})
+    return pairs
+
+
+def changed_dependencies(before, after):
+    """Dependencies added or whose specifier changed between two package.json texts.
+
+    Parameters: before (str or None) text before the edit, None when the file did not exist; after (str) text after.
+    Returns: list of (name, specifier); empty when `after` is not valid JSON (the user is mid-edit).
+    Raises: DependencyCheckError when `before` exists but is not valid JSON.
+    """
+    now = _tables(after)
+    if now is None:
+        return []
+    if before is None or not before.strip():
+        old = {}
+    else:
+        old = _tables(before)
+        if old is None:
+            raise DependencyCheckError("current package.json is not valid JSON")
+    return [pair for pair in now if pair not in old]
+
+
+def queried_version(specifier):
+    """The version to ask OSV about for a specifier.
+
+    Parameters: specifier (str) the value in the dependency table.
+    Returns: str, the exact version or the lower bound of a ^, ~, >= or > range; None when the specifier cannot be
+    reduced safely (tags, `*`, URLs, file:, git, workspace:, npm: aliases, x-ranges, compound ranges).
+    """
+    match = _SPECIFIER_RE.fullmatch(specifier.strip())
+    if not match:
+        return None
+    operator, major, minor, patch, tail = match.groups()
+    parts = [major, minor, patch]
+    if operator in ("^", "~", ">=", ">"):
+        return ".".join(part or "0" for part in parts) + (tail or "")
+    if None in parts:
+        return None
+    return ".".join(parts) + (tail or "")
+
+
+def _load_cache():
+    """The cache as {"name@version": {"ids": [...], "at": epoch}}; empty when the file does not exist.
+
+    Raises: DependencyCheckError when the file exists and is unreadable, not JSON or not an object (never reset).
+    """
+    if not os.path.exists(CACHE_PATH):
+        return {}
+    try:
+        with open(CACHE_PATH, encoding="utf-8") as stream:
+            data = json.load(stream)
+    except (OSError, ValueError) as exc:
+        raise DependencyCheckError(f"advisory cache {CACHE_PATH} is unreadable: {exc}") from exc
+    if not isinstance(data, dict):
+        raise DependencyCheckError(f"advisory cache {CACHE_PATH} is not a JSON object")
+    return data
+
+
+def _fresh_ids(entry, now):
+    """The cached IDs of an entry, or None when it is missing, malformed or older than CACHE_TTL_S."""
+    if not isinstance(entry, dict) or not isinstance(entry.get("ids"), list) or not isinstance(entry.get("at"), (int, float)):
+        return None
+    return entry["ids"] if now - entry["at"] < CACHE_TTL_S else None
+
+
+def _advisories(queries):
+    """Advisory IDs for (name, version) pairs, from the cache and one OSV batch for the rest.
+
+    Parameters: queries (list of (name, version)).
+    Returns: tuple (list of IDs or None per query in order, str cache-write problem or "").
+    Raises: DependencyCheckError when the cache is corrupt or OSV cannot answer.
+    """
+    cache, now = _load_cache(), time.time()
+    found = {query: _fresh_ids(cache.get("@".join(query)), now) for query in queries}
+    missing = [query for query in queries if found[query] is None]
+    if not missing:
+        return [found[query] for query in queries], ""
+    try:
+        answered = freshness.npm_advisories_batch(missing, OSV_TIMEOUT_S)
+    except RuntimeError as exc:
+        raise DependencyCheckError(str(exc)) from exc
+    for query, ids in zip(missing, answered):
+        found[query] = ids
+        if ids is not None:
+            cache["@".join(query)] = {"ids": ids, "at": now}
+    cache = {key: entry for key, entry in cache.items() if _fresh_ids(entry, now) is not None}
+    try:
+        os.makedirs(os.path.dirname(CACHE_PATH), exist_ok=True)
+        atomic_io.write_secret_text(CACHE_PATH, json.dumps(cache, indent=1))
+    except OSError as exc:
+        return [found[query] for query in queries], f"advisory cache not saved: {exc}"
+    return [found[query] for query in queries], ""
+
+
+def notice(before, after):
+    """The dependency notice for one package.json edit.
+
+    Parameters: before (str or None) file text before the edit; after (str) text after it.
+    Returns: str, "Smart Tool dependencies: ..." lines for the changed dependencies that have advisories or could not
+    be verified, "" when there is nothing to say.
+    Raises: DependencyCheckError when the check could not run.
+    """
+    changed = changed_dependencies(before, after)
+    handled, extra = changed[:MAX_PACKAGES], len(changed) - MAX_PACKAGES
+    lines, queries = {}, []
+    for name, specifier in handled:
+        version = queried_version(specifier) if freshness.NPM_NAME_RE.fullmatch(name) else None
+        if version is None:
+            lines[(name, specifier)] = f"`{name}@{specifier}` specifier not verified."
+        else:
+            queries.append((name, specifier, version))
+    problem = ""
+    if queries:
+        ids, problem = _advisories([(name, version) for name, _specifier, version in queries])
+        for (name, specifier, version), advisories in zip(queries, ids):
+            label = f"`{name}@{specifier}`" + ("" if version == specifier else f" (checked {version})")
+            if advisories is None:
+                lines[(name, specifier)] = f"{label} advisory check incomplete, not verified."
+            elif advisories:
+                shown = ", ".join(advisories[:MAX_IDS_SHOWN])
+                more = f" and {len(advisories) - MAX_IDS_SHOWN} more" if len(advisories) > MAX_IDS_SHOWN else ""
+                lines[(name, specifier)] = f"{label} has advisory IDs {shown}{more}."
+    parts = [lines[pair] for pair in handled if pair in lines]
+    if extra > 0:
+        parts.append(f"{extra} more not checked.")
+    if problem and parts:
+        parts.append(f"({problem})")
+    return PREFIX + " ".join(parts) if parts else ""

package/document_text.py

@@ -1,10 +1,17 @@
-"""Extração local de texto DOCX, sem Office, execução de macros ou acesso externo."""
+"""Extração local de texto DOCX e PDF, sem Office, execução de macros ou acesso externo."""
 import io
+import json
 import re
+import subprocess
+import sys
 import zipfile
 import xml.etree.ElementTree as ET
 
-VERSION = 1
+VERSION = 2
+PDF_MAX_BYTES = 20 * 1024 * 1024
+PDF_TIMEOUT_S = 20
+PDF_MAX_PAGES = 300
+PDF_MAX_CHARS = 1_000_000
 MAX_ARCHIVE_BYTES = 10 * 1024 * 1024
 MAX_XML_BYTES = 8 * 1024 * 1024
 MAX_TEXT_BYTES = 1024 * 1024
@@ -15,6 +22,20 @@ class DocumentError(ValueError):
     pass
 
 
+class PdfTimeout(DocumentError):
+    pass
+
+
+class PdfEncrypted(DocumentError):
+    pass
+
+
+class PdfUnreadable(DocumentError):
+    def __init__(self, message, detail):
+        super().__init__(message)
+        self.detail = detail
+
+
 class _DocumentTreeBuilder(ET.TreeBuilder):
     def doctype(self, name, pubid, system):
         # O callback também recusa DTD em XML UTF-16, antes de expandir entidades.
@@ -65,5 +86,95 @@ def extract_docx(data):
         raise DocumentError('Could not extract text from this DOCX.') from exc
 
 
+PDF_SCRIPT = """
+import io, json, sys
+import pypdf
+reader = pypdf.PdfReader(io.BytesIO(sys.stdin.buffer.read()))
+if reader.is_encrypted and reader.decrypt("") == pypdf.PasswordType.NOT_DECRYPTED:
+    print(json.dumps({"encrypted": True}))
+    sys.exit()
+limit, budget = int(sys.argv[1]), int(sys.argv[2])
+pages, size = [], 0
+for page in reader.pages[:limit]:
+    pages.append((page.extract_text() or "").strip()[:budget - size])
+    size += len(pages[-1])
+    if size >= budget:
+        break
+print(json.dumps({"pages": pages, "count": len(reader.pages)}))
+"""
+
+
+def run_pdf(data, timeout, max_pages, max_chars):
+    """Text of a PDF (pypdf, pure Python), page by page, with `[page N]` markers. A hostile PDF can make pypdf parse for
+    minutes, so it runs in an isolated subprocess (`-I`) killed at the deadline.
+
+    Parameters: data (PDF bytes), timeout (seconds before the subprocess is killed), max_pages, max_chars (cap on the
+    text read, over all pages). Returns: the marked text, with a `[pages A-B not read]` line when a cap cut it; empty
+    pages are left out. Raises: PdfTimeout, PdfEncrypted (a password is needed), PdfUnreadable (corrupt; `detail` has
+    the last line pypdf reported), all DocumentError."""
+    try:
+        done = subprocess.run([sys.executable, "-I", "-c", PDF_SCRIPT, str(max_pages), str(max_chars)],
+                              input=data, capture_output=True, timeout=timeout,
+                              creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0))
+    except subprocess.TimeoutExpired:
+        raise PdfTimeout(f"PDF not read in {timeout:.0f}s.") from None
+    if done.returncode:
+        detail = (done.stderr.decode("utf-8", "replace").strip().splitlines() or ["no output"])[-1]
+        raise PdfUnreadable("PDF could not be read.", detail)
+    try:
+        result = json.loads(done.stdout)
+        if result.get("encrypted"):
+            raise PdfEncrypted("PDF protected by a password.")
+        pages, count = result["pages"], result["count"]
+    except (ValueError, KeyError, AttributeError) as exc:
+        if isinstance(exc, PdfEncrypted):
+            raise
+        raise PdfUnreadable("PDF could not be read.", f"unexpected extractor output ({type(exc).__name__})") from None
+    text = "\n\n".join(f"[page {number}]\n{page}" for number, page in enumerate(pages, 1) if page)
+    if count > len(pages):
+        text += f"\n\n[pages {len(pages) + 1}-{count} not read]"
+    return text
+
+
+def extract_pdf(data):
+    """Indexable text of a PDF through run_pdf with the indexer's deadline and caps.
+
+    Parameters: data (PDF bytes). Returns: UTF-8 bytes of the marked text. Raises: DocumentError with a readable reason
+    for a timeout, a password, a corrupt file or a PDF without a text layer (scanned; there is no OCR)."""
+    try:
+        text = run_pdf(data, PDF_TIMEOUT_S, PDF_MAX_PAGES, PDF_MAX_CHARS)
+    except PdfUnreadable as exc:
+        raise PdfUnreadable(f"PDF could not be read ({exc.detail})"[:200], exc.detail) from None
+    if "[page " not in text:
+        raise DocumentError("PDF has no text layer (scanned pages are not read).")
+    return (text + "\n").encode("utf-8", "replace")
+
+
+def is_extracted(path):
+    """Whether the file type goes through a text extractor before indexing.
+
+    Parameters: path (any file name or path). Returns: True for `.docx` and `.pdf`, case-insensitive."""
+    return str(path).lower().endswith(('.docx', '.pdf'))
+
+
+def size_limit(path):
+    """Largest file size, in bytes, the extractor of this type accepts.
+
+    Parameters: path of an extracted type (see is_extracted). Returns: the byte limit of its extractor."""
+    return PDF_MAX_BYTES if str(path).lower().endswith('.pdf') else MAX_ARCHIVE_BYTES
+
+
+def extract(path, data):
+    """Text of an extracted-type file.
+
+    Parameters: path (decides the type, see is_extracted), data (file bytes). Returns: UTF-8 text bytes. Raises:
+    DocumentError with a readable reason."""
+    return extract_pdf(data) if str(path).lower().endswith('.pdf') else extract_docx(data)
+
+
 def location_kind(path):
-    return 'extracted_text_line' if str(path).lower().endswith('.docx') else 'source_line'
+    """Kind of line numbers a result of this file carries.
+
+    Parameters: path. Returns: `extracted_text_line` (lines of the extracted text, not of the file) for extracted
+    types, else `source_line`."""
+    return 'extracted_text_line' if is_extracted(path) else 'source_line'

package/dossier.py

@@ -16,7 +16,7 @@ TITLE_CHARS = 80
 MAX_FILE_BYTES = 512 * 1024
 BACKLOG_FILES = ("docs/BACKLOG.md", "BACKLOG.md")
 INSTRUCTION_FILES = (("CLAUDE.md", "claude_md"), (".claude/CLAUDE.md", "claude_md"), ("AGENTS.md", "agents_md"))
-_WORD = re.compile(r"[\w]{3,}", re.UNICODE)
+_WORD = re.compile(r"[\w]{2,}", re.UNICODE)
 _STOPWORDS = {
     "the", "and", "for", "how", "what", "where", "which", "does", "with", "from", "that", "this", "are", "when", "who",
     "why", "into", "used", "use", "uses", "its", "has", "have", "code", "project", "file", "files",
@@ -24,13 +24,15 @@ _STOPWORDS = {
     "pela", "ele", "ela", "isso", "esse", "essa", "este", "esta", "fica", "ficam", "usa", "usado", "usada", "sobre",
     "entre", "depois", "antes", "quando", "cada", "seu", "sua", "seus", "suas", "não", "sem", "mais", "código",
     "projeto", "arquivo", "arquivos", "decide", "funciona",
+    "is", "in", "to", "of", "on", "at", "by", "an", "as", "it", "or", "be", "do", "if", "so", "we", "no", "up", "me", "my",
+    "he", "us", "am", "de", "da", "em", "na", "os", "um", "se", "ao", "ou", "eu", "te", "é", "já", "só",
 }
 _UNIT_START = re.compile(r"#{1,6}\s|[-*]\s")
 
 
 def focus_terms(*texts):
-    """Casefolded content words of the texts (3+ characters, common question words dropped), camelCase split, in
-    order: what a window is scored by."""
+    """Casefolded content words of the texts (2+ characters, common question and function words dropped), camelCase
+    split, in order: what a window is scored by."""
     found = []
     for text in texts:
         spaced = re.sub(r"([a-z])([A-Z])", r"\1 \2", text or "")

package/freshness.py

@@ -62,44 +62,83 @@ def npm_registry_evidence(package):
     try:
         with urllib.request.urlopen(request, timeout=7) as response:
             data = json.load(response)
-        version = data.get("version") if isinstance(data, dict) else None
-        if not isinstance(version, str) or not version or len(version) > 80:
+        latest = data.get("version") if isinstance(data, dict) else None
+        if not isinstance(latest, str) or not latest or len(latest) > 80:
             return None
         return {"title": f"npm registry: {package}", "url": url,
-                "snippet": f"Latest version published on the npm registry: {version}",
-                "tier": "npm_registry", "npm_package": package, "npm_version": version}
+                "snippet": f"Latest version published on the npm registry: {latest}",
+                "tier": "npm_registry", "npm_package": package, "npm_version": latest}
     except Exception:
         return None
 
 
-def npm_advisories(package, version):
-    """IDs conhecidos pela OSV para a versão; None significa checagem indisponível."""
-    if (not package or not version
-            or not re.fullmatch(r"(?:@[a-z0-9._-]+/)?[a-z0-9._-]+", package)):
-        return None
-    body = json.dumps({"package": {"name": package, "ecosystem": "npm"},
-                       "version": version}).encode("utf-8")
-    request = urllib.request.Request(
-        "https://api.osv.dev/v1/query", data=body, method="POST",
+NPM_NAME_RE = re.compile(r"(?:@[a-z0-9._-]+/)?[a-z0-9._-]+")
+
+
+def _osv_query(package, package_version):
+    """One OSV query object: the package (npm) and the version asked about."""
+    return {"package": {"name": package, "ecosystem": "npm"}, "version": package_version}
+
+
+def _osv_request(endpoint, payload):
+    """The POST request to an OSV endpoint (`query` or `querybatch`) with the product's headers and User-Agent."""
+    return urllib.request.Request(
+        "https://api.osv.dev/v1/" + endpoint, data=json.dumps(payload).encode("utf-8"), method="POST",
         headers={"Content-Type": "application/json", "Accept": "application/json",
                  "User-Agent": version.user_agent(config.load_config().get("contact"))},
     )
+
+
+def _osv_ids(data):
+    """Sorted advisory IDs of one OSV result object; None when it is incomplete (next_page_token) or malformed, since
+    neither confirms the absence of a record."""
+    if isinstance(data, dict) and data.get("next_page_token"):
+        return None  # Página incompleta não confirma ausência de aviso novo.
+    rows = data.get("vulns", []) if isinstance(data, dict) else None
+    if not isinstance(rows, list):
+        return None
+    ids = [row.get("id") for row in rows if isinstance(row, dict)]
+    if any(not isinstance(advisory, str) or not advisory for advisory in ids):
+        return None
+    return sorted(set(ids))
+
+
+def npm_advisories(package, package_version):
+    """IDs conhecidos pela OSV para a versão; None significa checagem indisponível."""
+    if not package or not package_version or not NPM_NAME_RE.fullmatch(package):
+        return None
+    request = _osv_request("query", _osv_query(package, package_version))
     try:
         with urllib.request.urlopen(request, timeout=7) as response:
-            data = json.load(response)
-        if isinstance(data, dict) and data.get("next_page_token"):
-            return None  # Página incompleta não confirma ausência de aviso novo.
-        rows = data.get("vulns", []) if isinstance(data, dict) else None
-        if not isinstance(rows, list):
-            return None
-        ids = [row.get("id") for row in rows if isinstance(row, dict)]
-        if any(not isinstance(advisory, str) or not advisory for advisory in ids):
-            return None
-        return sorted(set(ids))
+            return _osv_ids(json.load(response))
     except Exception:
         return None
 
 
+def npm_advisories_batch(packages, timeout):
+    """Advisory IDs for each (package, version) of `packages` from one OSV `querybatch` call.
+
+    Parameters: packages (list of (name, version)); timeout (float) seconds for the request.
+    Returns: list in the same order, each the sorted ID list or None when that package's answer is incomplete
+    (next_page_token) or malformed, so it is unverified and not clean.
+    Raises: RuntimeError when the service cannot be reached, answers badly or answers a different number of results.
+    """
+    if any(not NPM_NAME_RE.fullmatch(name) or not package_version for name, package_version in packages):
+        raise ValueError(f"Invalid npm package query: {packages!r}")
+    request = _osv_request("querybatch", {"queries": [_osv_query(name, package_version)
+                                                      for name, package_version in packages]})
+    try:
+        with urllib.request.urlopen(request, timeout=timeout) as response:
+            data = json.load(response)
+    except Exception as exc:
+        raise RuntimeError(f"OSV querybatch failed: {exc}") from exc
+    results = data.get("results") if isinstance(data, dict) else None
+    if not isinstance(results, list) or len(results) != len(packages):
+        raise RuntimeError(f"OSV querybatch answered {len(results) if isinstance(results, list) else 'no'} results "
+                           f"for {len(packages)} queries")
+    return [_osv_ids(result) for result in results]
+
+
 def npm_osv_evidence(package, version):
     ids = npm_advisories(package, version)
     if ids is None:

package/hook_decision.py

@@ -12,9 +12,11 @@ chamada não dá para montar. Nenhuma decisão aqui é sobre periculosidade, só
 """
 import os
 
+import affected_tests
 import code_graph
 import code_impact
 import config
+import dependency_notice
 import doc_check
 import duplicates
 import edit_preview
@@ -182,7 +184,16 @@ def _impact_line(root, rel, name):
     counts = found["counts"]
     tests = ", ".join(f"{t['function']} ({t['path']})" for t in found["tests"])
     calls = f"; tests calling it: {tests}" if tests else ""
-    return f" {counts['callers']} caller(s){calls}; project_manage affected_tests lists every test to run after the edit."
+    return (f" {counts['callers']} caller(s){calls}{_run_command(root, found['tests'])}; project_manage affected_tests "
+            "lists every test to run after the edit.")
+
+
+def _run_command(root, tests):
+    """`; run them: <command>` for the test files of `tests`, one command per runner and folder, or "" when no runner
+    is known for them."""
+    commands = affected_tests._commands(root, sorted({t["path"] for t in tests})) if tests else []
+    runnable = [f"cd {c['cwd']} && {c['command']}" if c["cwd"] != "." else c["command"] for c in commands if c["command"]]
+    return f"; run them: {' ; '.join(runnable)}" if runnable else ""
 
 
 def _doc_lines(findings):
@@ -215,19 +226,35 @@ def _duplicate_lines(findings):
             + " If the copy is intentional (moving the function, a deliberate fork), go ahead.")
 
 
-def _edit_review(payload, doc_mode, duplicate_mode):
+def _edit_review(payload, doc_mode, duplicate_mode, dependency_mode):
     """Checks of an edit before it runs. Docstrings follow doc_mode, in code files of registered projects only (tests,
     docs and files outside a project are left out, as in the docs coverage): missing ones deny the edit in require mode and become a reminder in remind mode; a
     documented function whose signature changed always gets a reminder. Copies of indexed functions follow
-    duplicate_mode and only add context, never deny."""
+    duplicate_mode and only add context, never deny. An edit of a package.json follows dependency_mode: advisories
+    of the added or changed dependencies are added as context, never a denial."""
     with code_graph.pending_scope():
-        return _edit_review_scoped(payload, doc_mode, duplicate_mode)
+        return _edit_review_scoped(payload, doc_mode, duplicate_mode, dependency_mode)
 
 
-def _edit_review_scoped(payload, doc_mode, duplicate_mode):
+def _dependency_part(before, after):
+    """The dependency notice of one package.json edit, or the "dependency check skipped" line when it cannot run.
+
+    Parameters: before (str or None) text before the edit; after (str) text after it.
+    Returns: str, empty when there is nothing to say.
+    """
+    try:
+        return dependency_notice.notice(before, after)
+    except dependency_notice.DependencyCheckError as exc:
+        return f"{dependency_notice.PREFIX}dependency check skipped: {exc}"
+
+
+def _edit_review_scoped(payload, doc_mode, duplicate_mode, dependency_mode):
     """`_edit_review` inside one `code_graph.pending_scope`, so the pending-changes scan runs once per review."""
-    doc_findings, duplicate_findings, failures = [], [], []
+    doc_findings, duplicate_findings, failures, dependency_parts = [], [], [], []
     for path, before, after in edit_preview.preview(payload.get("tool_name"), payload.get("tool_input"), payload.get("cwd")):
+        if dependency_mode != "off" and os.path.basename(path) == "package.json":
+            dependency_parts += [part for part in (_dependency_part(before, after),) if part]
+            continue
         if not doc_check.supported(path):
             continue
         root = _project_of(path)
@@ -242,7 +269,7 @@ def _edit_review_scoped(payload, doc_mode, duplicate_mode):
             except (ValueError, RuntimeError) as exc:
                 failures.append(f"Smart Tool duplicates check skipped: {exc}")
     parts = ([_doc_lines(doc_findings)] if doc_findings else []) + \
-        ([_duplicate_lines(duplicate_findings)] if duplicate_findings else []) + failures
+        ([_duplicate_lines(duplicate_findings)] if duplicate_findings else []) + failures + dependency_parts
     if not parts:
         return {}
     message = " ".join(parts)
@@ -253,16 +280,17 @@ def _edit_review_scoped(payload, doc_mode, duplicate_mode):
 
 def decide(payload, client, web_route, jobs):
     """Hook output for one PreToolUse payload: edits follow doc_mode (require | remind | off) and duplicate_mode
-    (warn | off), the other tools follow hook_mode (redirect | advise | off). In redirect mode the Smart Tool call that
+    (warn | off) and dependency_mode (warn | off), the other tools follow hook_mode (redirect | advise | off). In redirect mode the Smart Tool call that
     replaces a redirected one starts in `jobs` under the call's tool_use_id (see `_redirect`); only a search whose
     query cannot be built, or a client that sends no tool_use_id, is denied with the exact call to make, and never in a
     subagent whose tool list has no Smart Tool MCP tool: it has nothing to switch to."""
     cfg = config.load_config()
     if isinstance(payload, dict) and payload.get("tool_name") in EDIT_TOOLS:
-        doc_mode, duplicate_mode = config.doc_mode(cfg), config.duplicate_mode(cfg)
-        if doc_mode == "off" and duplicate_mode == "off":
+        doc_mode, duplicate_mode, dependency_mode = (config.doc_mode(cfg), config.duplicate_mode(cfg),
+                                                     config.dependency_mode(cfg))
+        if doc_mode == "off" and duplicate_mode == "off" and dependency_mode == "off":
             return {}
-        return _edit_review(payload, doc_mode, duplicate_mode)
+        return _edit_review(payload, doc_mode, duplicate_mode, dependency_mode)
     if isinstance(payload, dict) and str(payload.get("tool_name")).startswith(f"mcp__{SMART_TOOL_SERVER}__"):
         if payload["tool_name"] == SMART_SEARCH_TOOL:
             router.CLIENT.set(client)

package/index_profile.py

@@ -11,7 +11,7 @@ PROFILE_VERSION = 2
 KINDS = ("code", "test", "doc")
 RETRY_AFTER_S = 3600
 MAX_DIRS = 90
-PROSE = {"md", "txt", "docx", "rst", "adoc"}
+PROSE = {"md", "txt", "docx", "pdf", "rst", "adoc"}
 # Data and markup inside a documentation, design or data-dump folder are reports and mockups, not code (GitHub Linguist
 # treats docs/ as documentation; a design/ HTML or a probe's JSON result pushed hook_decision.py out of the code block).
 DOC_DATA = {"html", "htm", "json", "csv", "xml", "yaml", "yml", "svg"}

package/indexer.py

@@ -129,7 +129,7 @@ def read_indexable(root, rel_path):
         size = os.path.getsize(full)
         if name.endswith(".svg") and size > MAX_SVG_INDEXABLE_BYTES:
             return None, SKIP_GRAPHIC
-        limit = document_text.MAX_ARCHIVE_BYTES if name.endswith('.docx') else MAX_INDEXABLE_BYTES
+        limit = document_text.size_limit(name) if document_text.is_extracted(name) else MAX_INDEXABLE_BYTES
         if size > limit:
             return None, SKIP_TOO_LARGE
         with open(full, "rb") as f:
@@ -138,9 +138,9 @@ def read_indexable(root, rel_path):
         return None, SKIP_UNREADABLE
     if len(data) > limit:
         return None, SKIP_TOO_LARGE
-    if name.endswith('.docx'):
+    if document_text.is_extracted(name):
         try:
-            data = document_text.extract_docx(data)
+            data = document_text.extract(name, data)
         except document_text.DocumentError as exc:
             return None, str(exc)
     prefix = data[:_SNIFF_BYTES]
@@ -507,7 +507,7 @@ def _stat_candidate(root, rel):
             return None, SKIP_UNREADABLE
     except OSError:
         return None, SKIP_UNREADABLE
-    if stat.st_size > (document_text.MAX_ARCHIVE_BYTES if name.endswith('.docx') else MAX_INDEXABLE_BYTES):
+    if stat.st_size > (document_text.size_limit(name) if document_text.is_extracted(name) else MAX_INDEXABLE_BYTES):
         return None, SKIP_TOO_LARGE
     if name.endswith(".svg") and stat.st_size > MAX_SVG_INDEXABLE_BYTES:
         return None, SKIP_GRAPHIC
@@ -523,7 +523,7 @@ def _stable_read(root, rel):
         if data is None:
             return None, encoding, before
         after, _ = _stat_candidate(root, rel)
-        if after == before and (str(rel).lower().endswith('.docx') or len(data) == before[1]):
+        if after == before and (document_text.is_extracted(rel) or len(data) == before[1]):
             return data, encoding, before
     return None, "file changed while reading", None
 
@@ -621,7 +621,7 @@ def _reindex(root, included_paths, embed_batch, chunk_lines=CHUNK_LINES, on_prog
                 continue
             observed[rel] = stat
             force_file = any(rel.casefold() == p or rel.casefold().startswith(p + "/") for p in forced)
-            extractor_changed = rel.lower().endswith('.docx') and meta.get('document_extractor_version') != document_text.VERSION
+            extractor_changed = document_text.is_extracted(rel) and meta.get('document_extractor_version') != document_text.VERSION
             if (full_check or building or force_file or extractor_changed or old_state.get(rel) != stat):
                 candidates.append(rel)
             else:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@allansantos-dev/smart-tool",
-  "version": "0.9.38",
+  "version": "0.9.40",
   "description": "Local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharper tools than their built-in search. Windows.",
   "license": "Apache-2.0",
   "author": "Allan Santos",

package/scout.py

@@ -12,6 +12,7 @@ Market pattern copied (docs/research/2026-10-24-scout-modelo-leve-no-smart-tool.
 excerpts, never an agent prompt, so no agent context is reopened per call.
 """
 import os
+import re
 
 import document_text
 import index_scope
@@ -35,6 +36,8 @@ PLAN_MAX_FOLDERS = 6
 OUTLINE_MAX_DIRS = 300
 OUTLINE_MAX_ROOT_FILES = 200
 _FIELDS = {"project_root", "question", "terms", "max_files"}
+DECISION_NOTES_MAX = 3
+_INTENT = re.compile(r"\b(why|por\s*qu[êe]|rationale|reason|motivo|raz[ãa]o|decid\w*|decis[ãa]o)\b", re.IGNORECASE)
 
 PLAN_SYSTEM = (
     "You plan a keyword scan of a code project that has no search index yet. Given a question and the project's "
@@ -91,9 +94,15 @@ def arguments(raw):
     return {"project_root": root, "question": question.strip(), "terms": (terms or "").strip(), "max_files": max_files}
 
 
+def is_intent_question(question):
+    """The question asks why something is as it is (why, reason, rationale, decided; por que, motivo, decisão): the
+    answer may sit in the notes the project keeps about its own intent, not in the code."""
+    return bool(_INTENT.search(question or ""))
+
+
 def disk_lines(root, path, start, end):
     """`[(line_number, text)]` of lines `start..end` of the file as it is on disk now, or None when the file cannot be
-    read by the index rules (secret, dotfile, too large, binary, gone) or holds no line text (docx)."""
+    read by the index rules (secret, dotfile, too large, binary, gone) or holds no line text (docx, pdf)."""
     if document_text.location_kind(path) != "source_line":
         return None
     data, encoding = indexer.read_indexable(root, path)
@@ -250,12 +259,13 @@ def valid_answer(raw, items):
     return raw["found"], answer, citations
 
 
-def render(question, source, items, citations=None, answer=None, found=None, model_note="", warnings=(), others=()):
+def render(question, source, items, citations=None, answer=None, found=None, model_note="", warnings=(), others=(),
+           notes=()):
     """The text returned to the agent. Lexical mode (`citations` None): every evidence window, within
     LITERAL_BUDGET_CHARS, after the model's answer when there is one (a model that cited nothing never leaves the
     answer without lines). Model mode: the model's answer, then only the cited ranges (sliced from the windows read
     from disk), and the uncited excerpts as `path:lines` pointers. `others` are candidates not shown, listed as
-    pointers."""
+    pointers. `notes` are the dossier hits for a question about intent: ids with one excerpt each, after the evidence."""
     out = [f"question: {question}", f"source: {source}", f"model: {model_note or 'none (literal lines only)'}"]
     if answer is not None:
         out.append(("answer: " if found else "answer (not found in the excerpts): ") + answer)
@@ -280,6 +290,9 @@ def render(question, source, items, citations=None, answer=None, found=None, mod
                 and citations is not None] + list(others)
     if pointers:
         out.append("other candidates (not shown): " + ", ".join(pointers))
+    if notes:
+        out.append("decision notes (project notes, not code):")
+        out += [f"- [{note['id']}] {note['source']}: {note['title']}: {note['snippet']}" for note in notes]
     for warning in warnings:
         out.append(f"warning: {warning}")
     return "\n".join(out) + "\n"

package/setup_audit.py

@@ -865,13 +865,55 @@ def _save_locked(report):
     atomic_io.write_secret_text(last_path, json.dumps(report, indent=1, ensure_ascii=False))
     changes = report.get("changes") or {}
     data = index()
-    data[index_key(report["project"])] = {"summary": report["summary"], "new": len(changes.get("new", [])),
-                                          "resolved": len(changes.get("resolved", [])),
-                                          "checked_at": report["checked_at"], "trigger": report["trigger"],
-                                          "snapshot_key": report["snapshot_key"]}
+    key = index_key(report["project"])
+    entry = {"project": report["project"], "summary": report["summary"], "new": len(changes.get("new", [])),
+             "resolved": len(changes.get("resolved", [])), "checked_at": report["checked_at"],
+             "trigger": report["trigger"], "snapshot_key": report["snapshot_key"],
+             "pending": _pending_notice(data.get(key), report)}
+    data[key] = entry
     _write_index(data)
 
 
+def _pending_notice(previous, report):
+    """The new errors and warnings of a scheduled run not yet told to the user, added to those a former run left
+    unannounced (by finding id). A manual run adds none: the user is looking at its result."""
+    pending = list((previous or {}).get("pending") or [])
+    if report["trigger"] == "schedule":
+        known = {item["id"] for item in pending}
+        pending += [{"id": f["id"], "severity": f["severity"], "check": f["check"], "subject": f["subject"]}
+                    for f in report["findings"]
+                    if f.get("new") and f["severity"] in ("error", "warn") and f["id"] not in known]
+    return pending
+
+
+def notices_due():
+    """[{project, errors, warnings, items}] for the projects whose scheduled audits found new errors or warnings since
+    the last call; marks them announced, so each finding is told once (the tray toast)."""
+    with _INDEX_LOCK:
+        data = index()
+        due = []
+        for entry in data.values():
+            pending = entry.get("pending") or []
+            if pending:
+                due.append({"project": entry["project"], "errors": sum(i["severity"] == "error" for i in pending),
+                            "warnings": sum(i["severity"] == "warn" for i in pending), "items": pending})
+                entry["pending"] = []
+        if due:
+            _write_index(data)
+    return due
+
+
+def toast_text(row):
+    """(title, lines) of the toast for one `notices_due` row."""
+    total = row["errors"] + row["warnings"]
+    title = f"Setup audit: {total} new problem{'s' if total != 1 else ''} in the agent configuration"
+    lines = [os.path.basename(row["project"].rstrip("\\/")) or row["project"]]
+    lines += [f"{item['severity']}: {item['check']} - {item['subject']}"[:120] for item in row["items"][:3]]
+    if len(row["items"]) > 3:
+        lines.append(f"and {len(row['items']) - 3} more")
+    return title, lines
+
+
 class Scheduler:
     """Periodic mode: audits every registered project whose folder exists, FIRST_RUN_DELAY_S after start and then
     every SCHEDULE_S, one project at a time, reusing MCP probes younger than PROBE_MAX_AGE_S. projects() returns the

package/setup_ui.py

@@ -201,11 +201,13 @@ def handle_health_get():
 
 
 def handle_notices():
-    """What the tray should show now: the model offer (once a day until answered) and models over the failure limit
-    (once a day each). Fetching marks them announced, so each toast is shown once per reminder period."""
+    """What the tray should show now: the model offer (once a day until answered), models over the failure limit
+    (once a day each) and the new errors or warnings of the scheduled setup audit (once per finding). Fetching marks
+    them announced, so each toast is shown once per reminder period."""
     upgrade = _checked_upgrade()
     offer = model_upgrade.due() if upgrade["state"] == "pending" else None
-    return {"upgrade": offer, "failing": model_health.alerts_due(), "check_error": upgrade["check_error"]}
+    return {"upgrade": offer, "failing": model_health.alerts_due(), "audit": setup_audit.notices_due(),
+            "check_error": upgrade["check_error"]}
 
 
 def handle_local_models_get():

package/smart_tool_daemon.py

@@ -529,7 +529,7 @@ TOOLS = [
         "description": (
             "Reads a known URL and returns only the answer to the prompt, written by the gateway's mini model - use instead of "
             "the native WebFetch. The page is cached for 24 h: another prompt on the same URL does not download it again. "
-            "JavaScript-built pages are rendered in the browser. Does not read localhost, internal networks or PDFs."
+            "JavaScript-built pages are rendered in the browser. Reads PDFs as text, page by page; does not read localhost or internal networks."
         ),
         "inputSchema": {
             "type": "object",
@@ -1706,6 +1706,11 @@ def _handle_smart_scout(arguments):
         others = list(dict.fromkeys(f"{row['path']}:{row['lines']}" for row in rows
                                     if row["path"] not in shown))[:SCOUT_MAX_POINTERS]
         source = _SCOUT_SOURCES[found["source"]]
+        notes = []
+        if scout.is_intent_question(question):
+            dossier_answer = dossier.query(root, query, max_hits=scout.DECISION_NOTES_MAX)
+            notes = dossier_answer["hits"]
+            warnings += [f"decision notes: {reason}" for reason in dossier_answer["reasons"]]
         text = None
         if model and items:
             calls += 1
@@ -1716,7 +1721,8 @@ def _handle_smart_scout(arguments):
                 found_answer, answer, citations = scout.valid_answer(raw, items)
                 note = f"{model} ({calls} call{'s' if calls > 1 else ''})"
                 if citations:
-                    text = scout.render(question, source, items, citations, answer, found_answer, note, warnings, others)
+                    text = scout.render(question, source, items, citations, answer, found_answer, note, warnings, others,
+                                        notes)
                 else:
                     shown_items = scout.evidence(root, rows, terms, max_files,
                                                  scout.LEXICAL_WINDOW_LINES)[:scout.UNCITED_WINDOWS]
@@ -1724,14 +1730,14 @@ def _handle_smart_scout(arguments):
                             if item["path"] not in {kept["path"] for kept in shown_items}]
                     text = scout.render(question, source, shown_items, None, answer, found_answer, note,
                                         warnings + ["the scout_model cited no lines; the best keyword windows are shown "
-                                                    "instead."], rest + others)
+                                                    "instead."], rest + others, notes)
             except Exception as exc:
                 warnings.append(f"scout_model {model} failed writing the answer ({_project_error(exc)}): literal lines "
                                 "only, no answer.")
                 failure = "answer"
                 items = scout.evidence(root, rows, terms, max_files, scout.LEXICAL_WINDOW_LINES)
         if text is None:
-            text = scout.render(question, source, items, warnings=warnings, others=others)
+            text = scout.render(question, source, items, warnings=warnings, others=others, notes=notes)
     _metric("scout", source=found["source"], model=bool(model) and failure is None, model_calls=calls, failure=failure,
             files=len({item["path"] for item in items}), chars=len(text), elapsed_s=round(time.monotonic() - started, 2))
     return _sanitize_text(text)

package/tray.py

@@ -22,6 +22,7 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 import autostart
 import config
 import daemon_launcher
+import setup_audit
 import toast_notice
 import update_check
 import version
@@ -114,7 +115,8 @@ def _poll_loop(icon):
 
 
 def _show_notices(icon, notices):
-    """Shows a toast for each pending notice: the new-default-models offer and each model over the failure limit."""
+    """Shows a toast for each pending notice: the new-default-models offer, each model over the failure limit and each
+    project whose scheduled setup audit found new errors or warnings."""
     offer = notices.get("upgrade")
     if offer:
         lines = [f"{role}: {change['current']} -> {change['new']}" for role, change in offer["changes"].items()]
@@ -129,6 +131,10 @@ def _show_notices(icon, notices):
             f"{row['model']} is failing", [f"{row['errors']} of {row['calls']} calls failed in 24 h as {row['role']}.", last[:160]],
             [("Switch model", toast_notice.action_uri("open-setup", section="models")),
              ("Mute 7 days", toast_notice.action_uri("mute-model", model=row["model"]))], tag="model-failing:" + row["model"])
+    for row in notices.get("audit") or []:
+        title, lines = setup_audit.toast_text(row)
+        toast_notice.show(title, lines, [("Open the audit", toast_notice.action_uri("open-setup", section="audit"))],
+                          tag="setup-audit:" + row["project"])
 
 
 def _notice_loop(icon):

package/version.py

@@ -1,7 +1,7 @@
 """Single source of the Smart Tool version and of the User-Agent sent to public APIs."""
 import re
 
-VERSION = "0.9.38"
+VERSION = "0.9.40"
 _CONTACT_RE = re.compile(r"^(?:[^\s@()<>;]+@[^\s@()<>;]+\.[^\s@()<>;]+|https?://[^\s()<>;]+)$")
 
 

package/web_fetch.py

@@ -20,14 +20,13 @@ import ipaddress
 import json
 import socket
 import ssl
-import subprocess
-import sys
 import threading
 import time
 import urllib.parse
 
 import trafilatura
 
+import document_text
 import model_client
 import research_cache
 
@@ -212,46 +211,19 @@ def thin(text):
     return len(stripped) < PLACEHOLDER_SCAN_CHARS and any(p in stripped[:600].lower() for p in PLACEHOLDERS)
 
 
-PDF_SCRIPT = """
-import io, json, sys
-import pypdf
-reader = pypdf.PdfReader(io.BytesIO(sys.stdin.buffer.read()))
-if reader.is_encrypted and reader.decrypt("") == pypdf.PasswordType.NOT_DECRYPTED:
-    print(json.dumps({"encrypted": True}))
-    sys.exit()
-limit, budget = int(sys.argv[1]), int(sys.argv[2])
-pages, size = [], 0
-for page in reader.pages[:limit]:
-    pages.append((page.extract_text() or "").strip()[:budget - size])
-    size += len(pages[-1])
-    if size >= budget:
-        break
-print(json.dumps({"pages": pages, "count": len(reader.pages)}))
-"""
-
-
 def pdf_text(body, url, deadline):
     """Text of a PDF (pypdf, pure Python), page by page up to PDF_MAX_PAGES. The native WebFetch reads PDFs and the hook
     sends it here, so a PDF used to be a dead end (9 times in 10 days of transcripts). A hostile PDF can make pypdf
     parse for minutes, so it runs in a subprocess killed at the request's deadline (at most PDF_TIMEOUT_S)."""
     timeout = min(PDF_TIMEOUT_S, _left(deadline))
     try:
-        done = subprocess.run([sys.executable, "-I", "-c", PDF_SCRIPT, str(PDF_MAX_PAGES), str(PAGE_STORE_MAX_CHARS)],
-                              input=body, capture_output=True, timeout=timeout,
-                              creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0))
-    except subprocess.TimeoutExpired:
+        return document_text.run_pdf(body, timeout, PDF_MAX_PAGES, PAGE_STORE_MAX_CHARS)
+    except document_text.PdfTimeout:
         raise FetchError(f"PDF not read in {timeout:.0f}s ({url}).") from None
-    if done.returncode:
-        detail = (done.stderr.decode("utf-8", "replace").strip().splitlines() or ["no output"])[-1]
-        raise FetchError(f"PDF could not be read ({url}): {detail}"[:300])
-    result = json.loads(done.stdout)
-    if result.get("encrypted"):
-        raise FetchError(f"PDF protected by a password is not read by web_fetch ({url}).")
-    pages = result["pages"]
-    text = "\n\n".join(f"[page {number}]\n{page}" for number, page in enumerate(pages, 1) if page)
-    if result["count"] > len(pages):
-        text += f"\n\n[pages {len(pages) + 1}-{result['count']} not read]"
-    return text
+    except document_text.PdfUnreadable as exc:
+        raise FetchError(f"PDF could not be read ({url}): {exc.detail}"[:300]) from None
+    except document_text.PdfEncrypted:
+        raise FetchError(f"PDF protected by a password is not read by web_fetch ({url}).") from None
 
 
 def read_page(url, deadline, render):