@allansantos-dev/smart-tool 0.9.43 → 0.9.45

package/CHANGELOG.md

@@ -1,5 +1,39 @@
 # Changelog
 
+## 0.9.45 - beta
+
+- The Playwright MCP server starts on first use. What the clients register is now `lazy_mcp_proxy.py` around
+  the real command: it answers `initialize`, `ping` and `tools/list` from the handshake the setup probe saved
+  (`playwright-mcp-handshake.json`) and starts the Node server on the first other request, replaying the client's
+  handshake to it, then passes everything through and kills the server's process tree when the client closes. Before, each
+  Claude session started its own Node server (about 97 MB of commit) whether or not it opened a browser. A missing or
+  broken cache fails loudly (exit 2, reason on stderr), never as an empty tool list; the heal at startup also checks the
+  real server's executable. Tests in `tests/test_lazy_mcp_proxy.py` and `tests/test_browser_control.py`.
+- Web searches record who asked. Each provider call (`web-search-health.jsonl`) and each research record
+  (`research-metrics.jsonl`) carries `requester` (`client`, `mcp_session`, `session`, `project`, `source`): the PreToolUse
+  hook, which gets the Claude Code session and folder, now also matches `web_search` and `web_fetch`, and the MCP call
+  claims that note by tool and arguments. Without the hook (Codex, or a hook not yet updated) only the client and the MCP
+  session are known and `source` is `none`. A redirected native WebSearch/WebFetch is attributed from the hook payload,
+  and `integration-metrics.jsonl` carries the same field. The query text is stored nowhere new. Tests in
+  `tests/test_requester.py`. The startup heal also rebuilds a registration that still starts the Node server directly or
+  whose handshake cache is gone, and the proxy gives up on a real server silent for 60 s on its first `initialize`.
+- The resident search browser's exit after 300 s without requests is now covered by a test that runs the real service
+  loop with a stub browser (`tests/test_browser_idle.py`); the value is unchanged.
+
+## 0.9.44 - beta
+
+- New tool `read_document`: a PDF, Word, PowerPoint or Excel file read as compact text, a range at a time. The answer opens
+  with the type, size, page, slide, sheet or part count and the estimated tokens of all the text, then the text of the
+  `range` asked for (`3`, `3-5`, `7-`, `-4`, `1,4-6`, a sheet name) up to `max_chars`, with a `next:` line to continue;
+  `info_only` returns just the facts. PDF pages are compacted (ligatures, hyphenated line ends, wrapped lines); a docx is
+  markdown (headings, lists, GFM tables, notes, headers) in parts of about 6,000 characters, since it has no fixed pages;
+  a slide carries its speaker notes; a sheet is one tab-separated line per row with cached formula values. A scanned PDF,
+  a password, a corrupt or hostile file and a range outside the document fail with the reason. Opt-in: the native Read is
+  not redirected and the tool is not loaded upfront (the server instructions say to load it with ToolSearch). No new
+  dependency; the extractors of the index now return their units (`pdf_pages`, `pptx_slides`, `xlsx_sheets`) and the
+  indexed text is unchanged. Sources and the measurement in `docs/research/2026-10-30-read-document.md`; tests in
+  `tests/test_read_document.py`.
+
 ## 0.9.43 - beta
 
 - The session-start audit report (Claude Code and Codex) is also given after a compaction and after a fork, the two

package/README.md

@@ -8,6 +8,7 @@ A local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharpe
 | `smart_scout` | Answers a question about the code with the lines that answer it: reads the candidate files inside Smart Tool and returns `path:lines` with the literal text, so the files never enter the agent's context; with a `scout_model`, a small model also writes a short answer and keeps only the lines that support it; a question about why something is as it is (why, reason, decided) also lists up to three decision notes from the notes the project keeps about its intent, as ids with one excerpt each |
 | `web_search` / `web_search_result` | Web search across several sources in parallel, in the query language and in English, with a cache shared across sessions; `depth="research"` runs a multi-round research loop |
 | `web_fetch` | Reads a known URL and returns only the answer to your prompt, written by a small model; pages are cached for 24 h. Pages over 60,000 characters are not cut: the model reads the parts closest to the prompt (needs an `embedding_model`) |
+| `read_document` | Reads a PDF, Word, PowerPoint or Excel file as compact text, a range at a time: type, size, page/slide/sheet count and estimated tokens first, then only the pages, slides or sheets asked for (`range="3-5"`), so the file never enters the agent's context whole. Opt-in: the native Read is not redirected. Scanned PDFs fail with that reason (no OCR) |
 | `project_manage` | Registers and indexes projects, shows coverage, code maps, duplicated functions and index storage; `graph` with `symbol` tells what changing a function touches: callers, calls, the tests that reach it and the files that import it |
 
 Agents that must click, type or log in also get Microsoft's [Playwright MCP](https://github.com/microsoft/playwright-mcp),
@@ -16,7 +17,7 @@ registered as the `playwright` server where this machine lets a browser be drive
 An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
 are cheaper, or just tells the agent what Smart Tool would do.
 
-**Status:** 0.9.43 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
+**Status:** 0.9.45 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
 
 ## How it works
 
@@ -149,7 +150,21 @@ Redirects) and Maintenance (Setup audit, Startup).
    `browser_run_code_unsafe`, which runs arbitrary JavaScript in the server process, is denied in the client
    (`permissions.deny` in `~/.claude/settings.json`, `disabled_tools` in `~/.codex/config.toml`); if it cannot be
    denied, the server is not registered. A `playwright` server you registered yourself is never touched. Remove it with
-   `claude mcp remove playwright -s user` / `codex mcp remove playwright`.
+   `claude mcp remove playwright -s user` / `codex mcp remove playwright`. The registered command is a small stdio
+   proxy (`lazy_mcp_proxy.py`, standard library only) that answers the handshake and the tool list from a cache the
+   probe writes and starts the Node server on the first tool call, so a session that never opens a browser does not
+   keep a 97 MB Node process; it ends with the session.
+
+   **Who asked for a web search**: every provider call in `~/.smart-tool/data/web-search-health.jsonl` and every
+   record of `research-metrics.jsonl` made for a `web_search` carries `requester`: `client`, `mcp_session` (first 12
+   characters of the daemon's session id), `session` and `project` (the Claude Code session id and folder) and `source`
+   (`hook`: the PreToolUse hook saw this call; `mcp-session`: an earlier call of the same MCP session was attributed;
+   `none`: only the client name is known, which is what Codex gives, since its hook does not cover MCP tools). The
+   query text is not stored there. The folder and session come from the installed hook: after an update, "Update hook"
+   on the setup page adds `web_search` and `web_fetch` to its matcher. A native WebSearch or WebFetch that the hook
+   redirects is attributed from the hook payload itself (`mcp_session` empty). `integration-metrics.jsonl` carries the
+   same `requester` on the web events. A `deep` crawl job writes no provider record, so it has none. The hook's note
+   waits up to 60 s for its MCP call; an identical query from two sessions at once can swap their notes.
 
    The hook behaviors:
    - **Redirect** (default): the call is not denied. Smart Tool answers it and the answer comes back as the output

package/browser_control.py

@@ -7,6 +7,10 @@ as the agent would: machines whose policy forbids it (corporate IT blocking remo
 keep today's read-only behavior, and the reason is shown in the setup screen instead of an error. A server named
 `playwright` the user already registered is left untouched.
 
+The registered command is lazy_mcp_proxy.py around the real server: every session starts the registered servers, and
+a Node process kept idle in each session that never opens a browser cost about 97 MB of commit each. The proxy answers
+the handshake from the cache the probe writes (HANDSHAKE_PATH) and starts the real server on the first tool call.
+
 Pages the agent opens are untrusted input. The server never exposes tools a page registers (WebMCP); every request
 the browser makes carries AGENT_HEADER, which the daemon refuses, so a page cannot steer the agent into Smart Tool's
 own setup page and its token (redirects included); `browser_run_code_unsafe` (arbitrary JavaScript in the server
@@ -18,6 +22,7 @@ import os
 import queue
 import re
 import subprocess
+import sys
 import tempfile
 import threading
 import time
@@ -26,6 +31,7 @@ import tomllib
 import atomic_io
 import endpoint_sync
 import install_runtime
+import lazy_mcp_proxy
 import node_locator
 import paths
 
@@ -34,6 +40,8 @@ CLI = install_runtime.PLAYWRIGHT_MCP_CLI
 STATUS_PATH = os.path.join(paths.DATA_DIR, "browser-control.json")
 OUTPUT_DIR = os.path.join(paths.DATA_DIR, "playwright-mcp")
 CONFIG_PATH = os.path.join(paths.DATA_DIR, "playwright-mcp.json")
+HANDSHAKE_PATH = os.path.join(paths.DATA_DIR, "playwright-mcp-handshake.json")
+PROXY = os.path.join(os.path.dirname(os.path.abspath(__file__)), "lazy_mcp_proxy.py")
 AGENT_HEADER = "X-Smart-Tool-Agent-Browser"
 OUTPUT_MAX_BYTES = 100 * 1024 * 1024
 PROBE_TIMEOUT_S = 60
@@ -68,6 +76,15 @@ def server_command():
             "--output-max-size", str(OUTPUT_MAX_BYTES)]
 
 
+def lazy_command(command):
+    """The command registered in the clients: lazy_mcp_proxy.py (console Python of this installation) around command,
+    the real server line, answering from HANDSHAKE_PATH until a tool call needs the real one."""
+    python = sys.executable
+    if os.path.basename(python).lower() == "pythonw.exe":
+        python = os.path.join(os.path.dirname(python), "python.exe")
+    return [python, PROXY, "--handshake", HANDSHAKE_PATH, "--", *command]
+
+
 def write_config():
     """Writes the Playwright MCP config file that adds AGENT_HEADER to every request of the browser (navigations,
     redirects, subresources; measured: no CORS preflight is added) and returns its path."""
@@ -77,14 +94,16 @@ def write_config():
     return CONFIG_PATH
 
 
-def probe(command, timeout=PROBE_TIMEOUT_S):
+def probe(command, timeout=PROBE_TIMEOUT_S, handshake_path=None):
     """Starts the server over stdio, opens about:blank and closes it, as an agent's first call would.
-    Returns None when the browser opened, else the reason it did not (a policy block, a crash, a timeout)."""
+    Returns None when the browser opened, else the reason it did not (a policy block, a crash, a timeout). With
+    handshake_path, a successful probe also saves the server's initialize result and tool list there, the cache
+    lazy_mcp_proxy answers from."""
     with tempfile.TemporaryDirectory(prefix="st-browser-probe-", ignore_cleanup_errors=True) as workdir:
-        return _probe(command, workdir, timeout)
+        return _probe(command, workdir, timeout, handshake_path)
 
 
-def _probe(command, workdir, timeout):
+def _probe(command, workdir, timeout, handshake_path=None):
     """probe() inside workdir, the server's working folder."""
     flags = getattr(subprocess, "CREATE_NO_WINDOW", 0)
     try:
@@ -118,14 +137,20 @@ def _probe(command, workdir, timeout):
                 return message["result"]
 
     try:
-        call(1, "initialize", {"protocolVersion": "2025-06-18", "capabilities": {},
-                               "clientInfo": {"name": "smart-tool-probe", "version": "1"}})
+        initialized = call(1, "initialize", {"protocolVersion": "2025-06-18", "capabilities": {},
+                                             "clientInfo": {"name": "smart-tool-probe", "version": "1"}})
         proc.stdin.write(json.dumps({"jsonrpc": "2.0", "method": "notifications/initialized"}) + "\n")
+        tools = call(4, "tools/list", {}).get("tools") if handshake_path else None
         result = call(2, "tools/call", {"name": "browser_navigate", "arguments": {"url": "about:blank"}})
         if result.get("isError"):
             text = " ".join(part.get("text", "") for part in result.get("content") or [])
             return f"browser did not open: {' '.join(text.split())[:300]}"
         call(3, "tools/call", {"name": "browser_close", "arguments": {}})
+        if handshake_path:
+            if not tools:
+                return "the server listed no tools"
+            os.makedirs(os.path.dirname(handshake_path), exist_ok=True)
+            atomic_io.write_secret_text(handshake_path, json.dumps({"initialize": initialized, "tools": tools}))
         return None
     except (OSError, RuntimeError, TimeoutError) as exc:
         return str(exc)
@@ -158,8 +183,9 @@ def registered():
     return found
 
 
-def registered_command(client):
-    """Executable the client's `playwright` server is registered with, or None when it is not registered."""
+def registered_executables(client):
+    """Executables the client's `playwright` server needs: the registered command and, behind lazy_mcp_proxy, the real
+    server's too; an empty list when it is not registered."""
     try:
         if client == "claude":
             with open(endpoint_sync.claude_json(), encoding="utf-8") as stream:
@@ -168,9 +194,32 @@ def registered_command(client):
             with open(_codex_config(), encoding="utf-8") as stream:
                 entry = (tomllib.loads(stream.read()).get("mcp_servers") or {}).get(SERVER)
     except (OSError, ValueError, AttributeError):
-        return None
+        return []
     command = entry.get("command") if isinstance(entry, dict) else None
-    return command if isinstance(command, str) else None
+    if not isinstance(command, str):
+        return []
+    args = entry.get("args") if isinstance(entry.get("args"), list) else []
+    real = args[args.index("--") + 1] if "--" in args and args.index("--") + 1 < len(args) else None
+    return [command] + ([real] if isinstance(real, str) else [])
+
+
+def _lazy_registration_usable(client):
+    """Whether the client's registration goes through lazy_mcp_proxy and the handshake cache it answers from is
+    readable: a registration of an older version (the Node server directly) or a lost cache both need ensure."""
+    try:
+        if client == "claude":
+            with open(endpoint_sync.claude_json(), encoding="utf-8") as stream:
+                entry = (json.load(stream).get("mcpServers") or {}).get(SERVER)
+        else:
+            with open(_codex_config(), encoding="utf-8") as stream:
+                entry = (tomllib.loads(stream.read()).get("mcp_servers") or {}).get(SERVER)
+        args = entry.get("args") or []
+        if PROXY not in args:
+            return False
+        lazy_mcp_proxy.load_handshake(HANDSHAKE_PATH)
+    except (OSError, ValueError, AttributeError, RuntimeError):
+        return False
+    return True
 
 
 def heal(log=None):
@@ -179,9 +228,10 @@ def heal(log=None):
     {client: ensure's outcome} for the clients refreshed."""
     refreshed = {}
     for client in ("claude", "codex"):
-        command = registered_command(client)
+        executables = registered_executables(client)
         ours = status().get(client, {}).get("state") in ("registered", "undenied")
-        if ours and command and not node_locator.exists(command):
+        if ours and executables and not (all(node_locator.exists(path) for path in executables)
+                                         and _lazy_registration_usable(client)):
             refreshed[client] = ensure(client, log)
     return refreshed
 
@@ -302,6 +352,12 @@ def _record(client, state, reason="", log=None):
     return data[client]
 
 
+def _clip(text, limit=300):
+    """text within limit characters, keeping its start and its end: a failed CLI call names the long command first and
+    the CLI's own message last."""
+    return text if len(text) <= limit else text[:limit // 3] + " ... " + text[-(limit - limit // 3 - 5):]
+
+
 def ensure(client, log=None):
     """Registers Playwright MCP in the client, or refreshes the registration Smart Tool made before (Node, Chromium
     and install paths change between versions). A `playwright` server the user registered is kept. When the browser
@@ -315,7 +371,7 @@ def ensure(client, log=None):
         return _record(client, "kept", "a playwright MCP server you registered is kept")
     try:
         command = server_command()
-        problem = probe(command)
+        problem = probe(command, handshake_path=HANDSHAKE_PATH)
     except (OSError, RuntimeError, subprocess.SubprocessError) as exc:
         problem = str(exc)
     try:
@@ -325,7 +381,7 @@ def ensure(client, log=None):
             if ours and client == "claude":
                 set_claude_deny(False)
             return _record(client, "unavailable", problem[:300], log)
-        endpoint_sync.run_cli(register_command(client, command))
+        endpoint_sync.run_cli(register_command(client, lazy_command(command)))
         if not registered().get(client):
             return _record(client, "unavailable", "the client CLI finished, but the server did not appear in its "
                            "configuration", log)
@@ -342,5 +398,5 @@ def ensure(client, log=None):
     except (OSError, RuntimeError, ValueError) as exc:
         # Our registration still in place after a failure (its removal failed) stays ours, flagged, never "kept".
         state = "undenied" if ours and registered().get(client) else "unavailable"
-        return _record(client, state, str(exc)[:300], log)
+        return _record(client, state, _clip(str(exc)), log)
     return _record(client, "registered")

package/client_hooks.py

@@ -29,7 +29,7 @@ HOME = os.path.expanduser("~")
 PREVIEW_TTL_S = 600
 CLIENTS = {
     "claude": {"label": "Claude Code", "names": {"claude-code"}, "files": [os.path.join(endpoint_sync.claude_dir(), "settings.json")],
-               "docs": "https://code.claude.com/docs/en/hooks", "event": "PreToolUse", "matcher": "Grep|Glob|Read|Bash|PowerShell|WebSearch|WebFetch|Edit|MultiEdit|Write|mcp__smart-tool__smart_search", "http": True,
+               "docs": "https://code.claude.com/docs/en/hooks", "event": "PreToolUse", "matcher": "Grep|Glob|Read|Bash|PowerShell|WebSearch|WebFetch|Edit|MultiEdit|Write|mcp__smart-tool__smart_search|mcp__smart-tool__web_search|mcp__smart-tool__web_fetch", "http": True,
                "post_matcher": "Grep|Bash|PowerShell|WebFetch|WebSearch"},
     "codex": {"label": "Codex", "prefix": "codex", "files": [os.path.join(endpoint_sync.codex_dir(), "hooks.json"), os.path.join(endpoint_sync.codex_dir(), "config.toml")],
               "docs": "https://learn.chatgpt.com/docs/hooks", "event": "PreToolUse", "matcher": "Bash|apply_patch",

package/document_read.py

@@ -0,0 +1,272 @@
+"""read_document: PDF, Word, PowerPoint and Excel files read as compact text, one page, slide or sheet range at a time.
+
+The agent gets the file facts first (type, size, pages, slides or sheets, estimated tokens) and then only the range it
+asks for, so a 200-page PDF is never loaded whole. Extraction is the one of document_text (pypdf in an isolated
+subprocess for PDF; zipfile and ElementTree with the shared guards for docx, pptx and xlsx); this module adds the unit
+split, the markdown shape, the compaction of PDF line wrapping and the range arithmetic."""
+import collections
+import math
+import os
+import re
+import threading
+import unicodedata
+
+import document_text
+
+KINDS = {'.pdf': 'pdf', '.docx': 'docx', '.pptx': 'pptx', '.xlsx': 'xlsx'}
+UNIT_NAMES = {'pdf': 'page', 'docx': 'part', 'pptx': 'slide', 'xlsx': 'sheet'}
+DEFAULT_MAX_CHARS = 20_000
+MAX_CHARS_LIMIT = 100_000
+DOCX_PART_CHARS = 6_000
+CHARS_PER_TOKEN = 4
+CACHE_DOCUMENTS = 4
+LONG_LINE = 60
+
+_cache = collections.OrderedDict()
+_cache_lock = threading.Lock()
+
+
+class Unit:
+    """One readable unit of a document: a PDF page, a docx part, a slide or a sheet."""
+
+    def __init__(self, number, text, name=''):
+        self.number = number
+        self.text = text
+        self.name = name
+
+
+class Document:
+    """A document split into units, with the facts the metadata block reports."""
+
+    def __init__(self, path, kind, size, units, facts):
+        self.path = path
+        self.kind = kind
+        self.size = size
+        self.units = units
+        self.facts = facts
+
+
+def estimate_tokens(text):
+    """Token estimate of a text: characters divided by four, rounded up (a rule of thumb, not a tokenizer count).
+
+    Parameters: text. Returns: the estimate as an int."""
+    return math.ceil(len(text) / CHARS_PER_TOKEN)
+
+
+def compact_pdf_text(text):
+    """PDF page text with the layout noise taken out: ligatures and compatibility forms normalised, a hyphen at a line
+    end joined to the word, a wrapped line (long, no closing punctuation, next one starting in lower case) joined to the
+    next, spaces collapsed and blank-line runs reduced to one.
+
+    Parameters: text (one page as pypdf returns it). Returns: the compact text."""
+    text = unicodedata.normalize('NFKC', text).replace('\r\n', '\n').replace('\r', '\n')
+    text = re.sub(r'(\w)-\n(?=[a-zà-ÿ])', r'\1', text)
+    lines = [re.sub(r'[ \t]+', ' ', line).strip() for line in text.split('\n')]
+    joined = []
+    for line in lines:
+        previous = joined[-1] if joined else ''
+        if (previous and len(previous) >= LONG_LINE and line[:1].islower()
+                and not previous.endswith(('.', ':', ';', '!', '?', ')', '|'))):
+            joined[-1] = previous + ' ' + line
+        else:
+            joined.append(line)
+    return re.sub(r'\n{3,}', '\n\n', '\n'.join(joined)).strip()
+
+
+def _pdf_units(data):
+    """Units and facts of a PDF, one unit per page; raises DocumentError for a password, a corrupt file or a file with no
+    text layer at all (scanned pages are not read, there is no OCR)."""
+    pages, count = document_text.pdf_pages(data, document_text.PDF_TIMEOUT_S, document_text.PDF_MAX_PAGES,
+                                           document_text.PDF_MAX_CHARS)
+    if not any(pages):
+        raise document_text.DocumentError(
+            f'PDF of {count} pages has no text layer (scanned pages are not read; there is no OCR).')
+    units = [Unit(number, compact_pdf_text(page) or '(no text on this page)') for number, page in enumerate(pages, 1)]
+    facts = {'pages': count, 'pages without text': sum(1 for page in pages if not page)}
+    if count > len(pages):
+        facts['not read'] = f'pages {len(pages) + 1}-{count} (over the {document_text.PDF_MAX_PAGES} page or ' \
+                            f'{document_text.PDF_MAX_CHARS:,} character cap)'
+    return units, facts
+
+
+def _docx_units(data):
+    """Units and facts of a docx: the markdown blocks grouped in parts of about DOCX_PART_CHARS characters, a block never
+    split (a docx has no fixed pages, they depend on the renderer)."""
+    parts, current, size = [], [], 0
+    for block in document_text.docx_blocks(data):
+        if current and size + len(block) > DOCX_PART_CHARS:
+            parts.append('\n'.join(current))
+            current, size = [], 0
+        current.append(block)
+        size += len(block) + 1
+    parts.append('\n'.join(current))
+    return [Unit(number, text) for number, text in enumerate(parts, 1)], {
+        'parts': len(parts), 'note': f'a docx has no fixed pages; a part is about {DOCX_PART_CHARS:,} characters'}
+
+
+def _pptx_units(data):
+    """Units and facts of a pptx: one unit per slide under `[slide N]`, the speaker notes under `[notes N]`."""
+    units = []
+    for number, body, notes in document_text.pptx_slides(data):
+        lines = [f'[slide {number}]'] + (body or ['(no text)']) + ([f'[notes {number}]'] + notes if notes else [])
+        units.append(Unit(number, '\n'.join(lines)))
+    return units, {'slides': len(units), 'slides without text': sum(1 for u in units if '\n(no text)' in u.text)}
+
+
+def _xlsx_units(data):
+    """Units and facts of an xlsx: one unit per sheet under `[sheet NAME]`, then one tab-separated line per row."""
+    units = []
+    for number, (name, rows) in enumerate(document_text.xlsx_sheets(data), 1):
+        units.append(Unit(number, '\n'.join([f'[sheet {name}]'] + (rows or ['(no text)'])), name))
+    return units, {'sheet names': ', '.join(f'{u.number}={u.name}' for u in units)}
+
+
+_BUILDERS = {'pdf': _pdf_units, 'docx': _docx_units, 'pptx': _pptx_units, 'xlsx': _xlsx_units}
+
+
+def load(path):
+    """The document at `path`, split into units. Parsed documents are kept for a few calls (keyed by path, size and
+    modification time), so reading page after page parses the file once.
+
+    Parameters: path (absolute path of a .pdf, .docx, .pptx or .xlsx file). Returns: a Document. Raises: ValueError for a
+    relative path, a missing file, an unsupported type or a file over the size limit of its type; DocumentError with a
+    readable reason for a file the extractor refuses (corrupt, encrypted, hostile, scanned, without text)."""
+    if not isinstance(path, str) or not path.strip():
+        raise ValueError("read_document requires 'path'")
+    if not os.path.isabs(path):
+        raise ValueError(f'read_document needs an absolute path, got {path!r}')
+    extension = os.path.splitext(path)[1].lower()
+    if extension not in KINDS:
+        raise ValueError(f"read_document reads {', '.join(KINDS)} files; {extension or 'this file'} is not supported "
+                         '(use the normal Read for plain text; legacy .doc, .xls and .ppt and macro files are not read)')
+    try:
+        info = os.stat(path)
+    except OSError as exc:
+        raise ValueError(f'cannot read {path}: {exc.strerror or type(exc).__name__}') from None
+    if not os.path.isfile(path):
+        raise ValueError(f'{path} is not a file')
+    if info.st_size > document_text.size_limit(path):
+        raise ValueError(f'{path} is {info.st_size / 1048576:.1f} MiB, over the '
+                         f'{document_text.size_limit(path) // 1048576} MiB limit of a {extension} file')
+    key = (os.path.normcase(path), info.st_size, info.st_mtime_ns)
+    with _cache_lock:
+        if key in _cache:
+            _cache.move_to_end(key)
+            return _cache[key]
+    with open(path, 'rb') as stream:
+        data = stream.read()
+    kind = KINDS[extension]
+    units, facts = _BUILDERS[kind](data)
+    document = Document(path, kind, info.st_size, units, facts)
+    with _cache_lock:
+        _cache[key] = document
+        while len(_cache) > CACHE_DOCUMENTS:
+            _cache.popitem(last=False)
+    return document
+
+
+def parse_range(spec, document):
+    """Unit numbers selected by a range text: `3`, `3-5`, `7-` (to the end), `-4` (the first four), a comma list of
+    those (`1,4-6`) and, for a workbook, a sheet name (case-insensitive).
+
+    Parameters: spec (the text), document (a Document). Returns: the sorted list of unit numbers. Raises: ValueError
+    naming the valid range for a malformed or out-of-range spec or an unknown sheet name."""
+    count = len(document.units)
+    unit = UNIT_NAMES[document.kind]
+    chosen = set()
+    for part in (piece.strip() for piece in spec.split(',')):
+        if document.kind == 'xlsx':
+            named = [u.number for u in document.units if u.name.lower() == part.lower()]
+            if named:
+                chosen.update(named)
+                continue
+        found = re.fullmatch(r'(\d*)\s*(-?)\s*(\d*)', part)
+        if not part or not found or not (found.group(1) or found.group(3)):
+            raise ValueError(f'range {spec!r} is not valid: use a {unit} number, a span like 3-5, 7- or -4, or a comma list'
+                             + ('; a sheet name also works' if document.kind == 'xlsx' else ''))
+        first, dash, last = found.groups()
+        if not dash and last:
+            raise ValueError(f'range {spec!r} is not valid: use a {unit} number, a span like 3-5, 7- or -4, or a comma list')
+        low = int(first) if first else 1
+        high = int(last) if last else (count if dash else low)
+        if low < 1 or high > count or low > high:
+            raise ValueError(f'range {spec!r} is outside the document: it has {count} {unit}s (1-{count})')
+        chosen.update(range(low, high + 1))
+    return sorted(chosen)
+
+
+def _spans(numbers):
+    """Compact text of unit numbers: `1-3, 7`."""
+    spans, start, previous = [], None, None
+    for number in numbers + [None]:
+        if start is None:
+            start = previous = number
+        elif number is not None and number == previous + 1:
+            previous = number
+        else:
+            spans.append(str(start) if start == previous else f'{start}-{previous}')
+            start = previous = number
+    return ', '.join(spans)
+
+
+def header(document):
+    """The metadata block that opens every answer: path, type, size, units and the estimated tokens of the whole text.
+
+    Parameters: document. Returns: the block as text."""
+    total = sum(estimate_tokens(unit.text) for unit in document.units)
+    lines = [f'document: {document.path}',
+             f'type: {document.kind} | size: {document.size / 1048576:.2f} MiB | '
+             f'{UNIT_NAMES[document.kind]}s: {len(document.units)} | est. tokens (all text): {total:,}']
+    lines += [f'{key}: {value}' for key, value in document.facts.items()
+              if value and key != UNIT_NAMES[document.kind] + 's']
+    return '\n'.join(lines)
+
+
+def read(path, range_spec=None, max_chars=DEFAULT_MAX_CHARS, info_only=False):
+    """The answer of read_document: the metadata block, then the chosen units as markdown, a note naming the next range
+    when the text was cut.
+
+    Parameters: path (see load), range_spec (see parse_range; None reads from the first unit up to max_chars),
+    max_chars (cap of the text returned, 1000-MAX_CHARS_LIMIT; a single unit over the cap is cut at a line with a
+    notice), info_only (only the metadata block). Returns: the text. Raises: as load and parse_range, ValueError for a
+    max_chars out of bounds."""
+    if isinstance(max_chars, bool) or not isinstance(max_chars, int) or not 1000 <= max_chars <= MAX_CHARS_LIMIT:
+        raise ValueError(f'max_chars must be an integer from 1000 to {MAX_CHARS_LIMIT}')
+    document = load(path)
+    head = header(document)
+    if info_only:
+        return head
+    numbers = parse_range(range_spec, document) if range_spec else list(range(1, len(document.units) + 1))
+    unit = UNIT_NAMES[document.kind]
+    body, shown, size, cut = [], [], 0, None
+    for number in numbers:
+        text = document.units[number - 1].text
+        label = f'[{unit} {number}]\n' if document.kind in ('pdf', 'docx') else ''
+        piece = label + text
+        if shown and size + len(piece) > max_chars:
+            break
+        if len(piece) > max_chars:
+            lines = piece.split('\n')
+            kept, used = [], 0
+            for line in lines:
+                if used + len(line) + 1 > max_chars and kept:
+                    break
+                kept.append(line)
+                used += len(line) + 1
+            piece = '\n'.join(kept)
+            cut = (f'{unit} {number} cut after {len(kept)} of {len(lines)} lines; raise max_chars '
+                   f'(up to {MAX_CHARS_LIMIT:,}) to read the rest')
+        body.append(piece)
+        shown.append(number)
+        size += len(piece) + 1
+        if cut:
+            break
+    returned = '\n\n'.join(body)
+    notes = [f'returned: {unit}s {_spans(shown)} (est. {estimate_tokens(returned):,} tokens)']
+    remaining = [number for number in numbers if number > shown[-1]]
+    if cut:
+        notes.append(cut)
+    if remaining:
+        notes.append(f'next: range="{remaining[0]}-{remaining[-1]}"' if not range_spec
+                     else f'not returned: {unit}s {_spans(remaining)} (max_chars); ask for them with range')
+    return '\n'.join([head] + notes + ['---', returned])